Local and Free Co-Pilot

Do you love the idea of GitHub Co-Pilot but don't want to spend $20/month on it?

With Ollama and Cody AI, you too can have a Co-Pilot-like experience without a bill; how awesome is that! The setup is pretty simple, and here it is high-level:

1. Install Ollama

2. Install Cody AI extension in VSCode

3. Do some configuration in Cody AI

4. Make you a better coder (or not)

Let's break this down and spell out the details. I will use this document from Sourcegraph, the creators of Cody AI.

Step 1: Install Ollama

This is pretty easy, go here and follow the install instructions. Once installed, you will need to launch it, and then in the command line for your OS (Google it if you don't know), run the following:

ollama pull <model-name>

Which model you use is up to you, Cody AI supports the following list:

1. Code Llama

2. StarCoder

3. DeepSeek Coder

For simplicity, use Code Llama, so the command would look like this:

ollama pull codellama

It should download that model. If you run into issues with memory or the size of the model (each model has different RAM needs, you may need to specify a specific model size). Rule of thumb is, the larger the model, the more RAM you need. So be prepared to read the details for each model on the Ollama page for the model you want to run. For example, for codellama, you would look at this page.

Then we need to start that model, again from the command line, execute the following:

ollama run codellama

Once you have the model running, you can test it by issuing a command on the command line. For instance, "code a snake game in Python".

Step 2: Install Cody VS Code extension

Head over to VS Code and search the marketplace for Cody VS Code extension, and install it. Once it's installed, we need to make a couple of tweaks to get it to work with local models.

Step 3: Configure Cody

Open up your command pallet in VS Code (command+shift+p on MacOS) and type in `Cody: Extension Settings`, scroll down till you find Cody > Autocomplete > Advanced: Provider, there should be a drop-down, select `experimental-ollama`.

Then we need to open the `settings.json` file and find the `cody.autocomplete.experimental.ollamaOptions` property. If you can't find how to open up the `settings.json` file, use the command palette in VS Code, and search for `Open User Settings (JSON)`, that will open the `settings.json` file in the editor. Look for the aforementioned section, and add/modify to show the following:

"cody.autocomplete.experimental.ollamaOptions": {
  "url": "http://localhost:11434",
  "model": "codellama"
}

This will point Cody AI to your locally running Ollama server, and format the queries to the right model. If you use a different port, or model, modify the above to suit your local needs.

To validate it's working correctly, open a file that has code, and start typing something, it should autocomplete the request. Alternatively, you can check the output tab in VS Code and you should see some calls to Ollama with status codes, the endpoint it's using, and the type of call.

Now pop open a cold beverage of your choice, you are now saving money each month.