Commit `7438a402ea`

7438a402eabd60b4accbbd651a4f8649942742af

parent: 0e208c8601

Unsigned

cmc <hello@cleberg.net> · 2024-11-01 17:38 UTC

add ollama continue post

Layout: unified · split

content/blog/2024-10-31-continue-ollama-code-assistant.org added +233

		@@ -0,0 +1,233 @@
	1	#+date: <2024-10-31 Thu 11:01:05>
	2	#+title: Using Ollama As a Code Assistant in VS Codium
	3	#+description: This post describes a starting point for using Ollama as a code assistant inside VS Codium (or VS Code).
	4	#+filetags: :ai:
	5	#+slug: continue-ollama-code-assistant
	6
	7	* Background
	8
	9	As someone who doesn't do software development full time and intends to actually
	10	enjoy it as a hobby, I've been interested in the /concept/ of code assistants
	11	for a while. However, I had a few issues:
	12
	13	1. I'm not a full-time developer, so I wanted to actually learn all the various
	14	features and rules of the projects I was building.
	15	2. The quality was quite low until recently.
	16	3. I wanted to use an open-source solution. Ideally, one with a free tier so
	17	that I could test it before paying for a subscription.
	18
	19	I recently discovered [[https://www.continue.dev/][Continue]], which allows for local LLM configurations via
	20	[[https://ollama.com/][Ollama]], so I've decided to try it out and see how helpful a code assistant can
	21	really be for someone like me, who doesn't do this professionally.
	22
	23	* Installation
	24
	25	Installation is a quick and painless process if you already have Ollama
	26	installed, but I'll assume that we're starting from scratch.
	27
	28	** Ollama
	29
	30	First, install Ollama for your system. For macOS devices, I'd recommend using
	31	Homebrew via the following command. However, you can also visit their website
	32	and install the software manually.
	33
	34	#+begin_src sh
	35	brew install ollama
	36	brew services start ollama
	37	#+end_src
	38
	39	Next, we will need to install the models requested by the Continue extension. By
	40	default, Continue asks you to install the =llama3.1:8b= and =starcoder2:3b=
	41	models. However, you can customize the configuration to specify different
	42	models, if preferred.
	43
	44	#+begin_src sh
	45	ollama pull llama3.1:8b
	46	ollama pull starcoder2:3b
	47	#+end_src
	48
	49	At this point, Ollama should be running on your local machine and have two
	50	models available for use.
	51
	52	You can test this by visiting =http://localhost:11434/= in your browser or use
	53	cURL in your shell:
	54
	55	#+begin_src sh
	56	curl http://localhost:11434
	57
	58	# If running, the response will look like this:
	59	# Ollama is running
	60	#+end_src
	61
	62	** Continue
	63
	64	To get started with using Ollama as a code assistant in VS Codium, you'll first
	65	need to install the Continue extension. This extension is available for free on
	66	the Visual Studio Code marketplace and can be easily installed via the built-in
	67	Extension view.
	68
	69	Follow these steps:
	70	1. Open VS Codium and navigate to the Extensions view by clicking on the square
	71	icon next to the file menu or pressing =Ctrl+Shift+X= (Windows/Linux) or
	72	=Cmd+Shift+X= (Mac).
	73	2. Search for "Continue" in the Extensions marketplace.
	74	3. Click on the Continue extension and click Install.
	75
	76	Once installed, restart VS Codium to ensure that the extension is properly
	77	loaded.
	78
	79	* Continue: Extension Configuration
	80
	81	When you first install Continue, it will ask which service you'll be using. If
	82	you select Ollama, it will check to make sure Ollama is running and you have
	83	installed the two default models from Ollama.
	84
	85	After this initial setup, you can open the =config.json= file by pressing =Cmd +
	86	p= and entering => Continue: Open config.json=. This will show a config such as
	87	the following:
	88
	89	#+begin_src json
	90	{
	91	"models": [
	92	{
	93	"title": "Llama 3.1 8B",
	94	"provider": "ollama",
	95	"model": "llama3.1:8b"
	96	},
	97	{
	98	"model": "claude-3-5-sonnet-20240620",
	99	"provider": "anthropic",
	100	"apiKey": "",
	101	"title": "Claude 3.5 Sonnet"
	102	}
	103	],
	104	"tabAutocompleteModel": {
	105	"title": "Starcoder 3b",
	106	"provider": "ollama",
	107	"model": "starcoder2:3b"
	108	},
	109	"customCommands": [
	110	{
	111	"name": "test",
	112	"prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
	113	"description": "Write unit tests for highlighted code"
	114	}
	115	],
	116	"contextProviders": [
	117	{
	118	"name": "code",
	119	"params": {}
	120	},
	121	{
	122	"name": "docs",
	123	"params": {}
	124	},
	125	{
	126	"name": "diff",
	127	"params": {}
	128	},
	129	{
	130	"name": "terminal",
	131	"params": {}
	132	},
	133	{
	134	"name": "problems",
	135	"params": {}
	136	},
	137	{
	138	"name": "folder",
	139	"params": {}
	140	},
	141	{
	142	"name": "codebase",
	143	"params": {}
	144	}
	145	],
	146	"slashCommands": [
	147	{
	148	"name": "edit",
	149	"description": "Edit selected code"
	150	},
	151	{
	152	"name": "comment",
	153	"description": "Write comments for the selected code"
	154	},
	155	{
	156	"name": "share",
	157	"description": "Export the current chat session to markdown"
	158	},
	159	{
	160	"name": "cmd",
	161	"description": "Generate a shell command"
	162	},
	163	{
	164	"name": "commit",
	165	"description": "Generate a git commit message"
	166	}
	167	],
	168	"embeddingsProvider": {
	169	"provider": "ollama",
	170	"model": "nomic-embed-text"
	171	}
	172	}
	173	#+end_src
	174
	175	You can modify this file with many different customizations. Refer to the
	176	[[https://docs.continue.dev/customize/config][Configuration options]] page for more information.
	177
	178	* Use Cases
	179
	180	While I'm sure there are a ton of use cases that I can't think of, I decided to
	181	test it out with this blog and some basic Python scripts I am currently using.
	182	Here are the most common ones I've used so far:
	183
	184	- Improving README documentation
	185	- Refactor my =salary_visualization.py= script to align with PEP8.
	186	- Auto-complete thoughts and suggest further ideas for topics in this blog post.
	187
	188	As an example, the following list of possible use cases was auto-generated by
	189	Continue:
	190
	191	- Auto-complete function names and variables: With Ollama enabled, typing a few
	192	characters into the editor will suggest matching functions or variables from
	193	the entire project.
	194	- Code suggestions for common tasks: Ollama can provide suggestions for common
	195	programming tasks, such as converting types or formatting code.
	196	- Live coding assistance: As you type, Ollama can offer live suggestions and
	197	corrections to help ensure your code is correct.
	198
	199	** Screenshots
	200
	201	Below are a few screenshots from my current VS Codium window:
	202
	203	#+caption: Available Continue Commands
	204	[[https://img.cleberg.net/blog/20241031-continue-ollama-code-assistant/continue_commands.png]]
	205
	206	#+caption: Continue Fullscreen Chat Window
	207	[[https://img.cleberg.net/blog/20241031-continue-ollama-code-assistant/continue_fullscreen.png]]
	208
	209	#+caption: Inline Hotkeys
	210	[[https://img.cleberg.net/blog/20241031-continue-ollama-code-assistant/continue_inline_hotkeys.png]]
	211
	212	#+caption: Inline Editing Suggestions
	213	[[https://img.cleberg.net/blog/20241031-continue-ollama-code-assistant/continue_inline.png]]
	214
	215	#+caption: Sidebar Context Window
	216	[[https://img.cleberg.net/blog/20241031-continue-ollama-code-assistant/continue_sidebar.png]]
	217
	218	* Conclusion
	219
	220	As it stands, it seems that the current iteration of code completion and review
	221	models from Ollama are quite good for my use case. In particular, it is able to
	222	suggest logical continuations of my thoughts in a blog post, generate accurate
	223	documentation based on my files, explain code to me with references within the
	224	project, and align my existing files to standards.
	225
	226	However, it is not perfect. I have noticed that it often goes off in a random
	227	direction, unrelated to the intent of what I'm writing (either blogging or
	228	programming). It also struggles to understand the full context without clear,
	229	specific, repeated instructions to refer to certain files, standards, etc. while
	230	suggesting improvements.
	231
	232	All together, I think it's useful enough to suggest as an add-on tool, but I
	233	would be highly skeptical of any suggestions it provides.

theme/templates/index.html +6 −6

		@@ -8,6 +8,12 @@ uid [ultimate] Christian Cleberg <hello@cleberg.net></pre>
8	8	<section>
9	9	<h2>Recent Blog Posts</h2>
10	10
	11	<div class="post">
	12	<time datetime="2024-10-31 11:01:05">2024-10-31</time>
	13	<a href="/blog/continue-ollama-code-assistant.html"
	14	>Using Ollama As a Code Assistant in VS Codium</a
	15	>
	16	</div>
11	17	<div class="post">
12	18	<time datetime="2024-09-23 19:52:20">2024-09-23</time>
13	19	<a href="/blog/self-hosting-transmission.html"
		@@ -20,12 +26,6 @@ uid [ultimate] Christian Cleberg <hello@cleberg.net></pre>
20	26	>Linux Observability with Self-Hosted Prometheus and Grafana Cloud</a
21	27	>
22	28	</div>
23		<div class="post">
24		<time datetime="2024-08-25 09:45:30">2024-08-25</time>
25		<a href="/blog/n8n-sentiment-analysis.html"
26		>Email Sentiment Analysis With n8n</a
27		>
28		</div>
29	29
30	30	<br />
31	31	<a href="/blog/">All Posts →</a>

Commit 7438a402ea

Commit `7438a402ea`