What if I tried self-hosting LLM Code Tooling?

What if I tried self-hosting LLM Code Tooling?

July 2025

Currently Code AI tooling is going through a weird cycle. We have the Windsurf x OpenAI deal falling through, Cursor with some interesting (bad) pricing changes, Claude Code being a loss leader – which leads me to think, so what would self-hosting this on a laptop look like?

Firstly, there are a lot of options which exist already, however the reality is that my using of AI auto-complete and agentic mode is fairly limited. With that in mind, I think my current setup is rooted mainly in the llama.cpp world of things.

For auto-complete I would probably stick with llama-vscode. For the agents and CLI world, aider.chat will probably suffice, though I found the options a bit unintuitive. There are of course other options like continue.dev or roo code, though my gut feel is that they feel too tied to the IDE for my liking. There are other CLI driven options, though whether or not they work nicely with llama.cpp is to be determined. Maybe more specifically, what I found helpful with aider was the slash commands. In general, my usage of AI tooling was:

• auto-complete
• /ask

And because of that, it means something lightweight is suffice. Infact aider is so lightweight, I don’t need to directly ‘install’ it. One key tip is to set the --map-tokens 1024 explicitly since it drops it for unknown models (note that 1k context is the default context size in aider anyway).

1# remember to run llama-serve -m path/to/code/model -c 0
2uvx --from aider-install aider \
3  --model openai/default \
4  --openai-api-base http://127.0.0.1:8080/ --openai-api-key NONE \
5  --map-tokens 1024 \
6  --no-show-model-warnings \
7  --no-gitignore

Of course it is probably better if its configured properly, but having the ability to just run llama-server ... and then quickly run a CLI command makes the setup might easier to reason with.