{"componentChunkName":"component---src-templates-post-js","path":"/blog/2024/06/07/ollama-continue/","result":{"data":{"post":{"html":"

Are you a developer who wants to use AI in their day to day coding but don’t have access to tools like GitHub Copilot, GitHub Copilot Workspace or Cloud based services like Azure OpenAI? If yes, read on and you’re going to have super powers by the end of this post.

\n\n

Why bother?

\n

If you’re wondering what is all the fuss and why should I bother, I have to tell you AI is moving like a speed train and if you don’t jump in, you’ll be left behind and it would be very hard to catch on later. AI can help you enhance your coding skills, increase your productivity, and help you learn a great deal more than you do in your job.

\n

Here are some reasons why any developer needs to use AI:

\n\n

Setup - let’s get cracking

\n

Enough talk, let’s get into the action. You will need these before you can leverage AI in your development environment:

\n\n

Install the code generation model

\n

Go into the environment which you have Ollama installed and install the starcoder model. You can get the 3b parameter version or if you’re adventurous like me, get the 7b version which is roughly 4GB.

\n\n \n Copy\n \n \n
ollama pull starcoder2:7b
\n

And you should see the model being pulled and installed.

\n

\n \n \n \n \n

\n

Install the extension

\n

First you need to install Continue extension in VS Code. You can click on the link from here or go into your VS Code extensions and get it installed.

\n

\n \n \n \n \n

\n

Configuring the extension

\n

Once you have both of those installed, you can go ahead and configure your extension to use your local Ollama instance. Open the config.json settings for Continue using Ctrl + Shift + P, then typing > continue: open config.json. Keep in mind that Continue supports a whole host of providers other than local ones. Go ahead and add the models you want to the config file. Do not forget to change the IP address to match your own instance if it is not available on localhost.

\n\n \n Copy\n \n \n
{\r\n  \"models\": [\r\n    {\r\n      \"title\": \"Llama 3\",\r\n      \"provider\": \"ollama\",\r\n      \"model\": \"llama3\",\r\n      \"apiBase\": \"http://localhost:11434/\"\r\n    },\r\n    {\r\n      \"title\": \"Starcoder 7b\",\r\n      \"provider\": \"ollama\",\r\n      \"model\": \"starcoder2:7b\",\r\n      \"apiBase\": \"http://localhost:11434/\"\r\n    }\r\n  ],\r\n  \"customCommands\": [\r\n    {\r\n      \"name\": \"test\",\r\n      \"prompt\": \"{{{ input }}}\\n\\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.\",\r\n      \"description\": \"Write unit tests for highlighted code\"\r\n    }\r\n  ],\r\n  \"tabAutocompleteModel\": {\r\n    \"title\": \"Starcoder 7b\",\r\n    \"provider\": \"ollama\",\r\n    \"model\": \"starcoder2:7b\",\r\n    \"apiBase\": \"http://localhost:11434/\"\r\n  },\r\n  \"allowAnonymousTelemetry\": true,\r\n  \"embeddingsProvider\": {\r\n    \"provider\": \"transformers.js\"\r\n  }\r\n}
\n

Testing

\n

Let’s test our setup, go ahead and open a file in which you have any code written. Select a code block and press Ctrl + L which opens the chat window, type Explain this code to me in a few sentences and press enter. You should see the explanation getting generated in the chat window:

\n

\n \n \n \n \n

\n

Using tab for auto completion

\n

You can use tab to complete code in your code file. Create a new file called index.js, enter below comments:

\n\n \n Copy\n \n \n
// a function which calculates fibonacci sequence\r\n// and returns the nth number in the sequence
\n

Go to the next line and wait until your see the function name getting generated, if it didn’t type the function name and wait, it should generate the function and show you in highlighted grey text:

\n

\n \n \n \n \n

\n

Use documentation as context for your questions

\n

You can use documentation when interacting with AI which adds more accuracy to the response. The team have done a fantastic job and have pretty much any language or framework you can think of linked. You can use the @docs and then select your preferred documentation, or just simply type the doc name if you know it.

\n

Let’s open the chat window using Ctrl + L, then type @react and then How do I update the page based on a parameter? then press Enter.

\n

\n \n \n \n \n

\n

Perform actions with slash

\n

You can kick off an action using the slash in the chat window, this could be edit, comment, share, commit, test (which I had in the config and shows you how to do custom ones) or any predefined custom command.

\n

\n \n \n \n \n

\n

I will use the test command, select the code in your file and press Ctrl + I, then /test and finally Enter.

\n

\n \n \n \n \n

\n

Use files as context

\n

You can use files in your context by using the @files tag. Simply type that in the search box and you will get prompted to select a file, or you can type the file name directly. Then type your question and press Enter.

\n

\n \n \n \n \n

\n

Understand errors in terminal

\n

When you get any errors or issues in your terminal, select the error text and press Ctrl + Shift + R and you will get an explanation:

\n

\n \n