LLM on NoRaincheck

Vibe Benchmarks

Sat, 04 Jul 2026 00:00:00 +0000

Vibe Benchmarks

July 2026

A quick round of informal benchmarking across a few local coding models. Nothing rigorous — just a few problems run through each model and seeing how they handled it. The goal was to get a sense of the trade-offs between quality, speed, and practical usability.

The models

Qwen Coder Next — the best quality by a fair margin. It understood the problem, produced clean code, and generally got it right on the first try. The problem is that it’s too large and too slow. The latency was noticeable, and the larger context windows it supports actually work against it — with more context comes more tokens to process, and the slowdown compounds.
Qwen 35B MoE — the best on balance. It matched Qwen Coder Next on most problems, was noticeably faster, and didn’t suffer from the same context-window bloat. For practical daily use, this is the sweet spot.
Cohere North Mini — extremely fast. Almost instant responses. But it failed to solve several problems that Qwen 35B solved after one or two turns. North Mini kept going in circles — same wrong approach, repeated, unable to course-correct. Speed is great when it works, but not much use if it can’t actually solve the problem.
Qwen 27B — too slow for my taste. I’d need to try it again under different conditions before forming a firm opinion. It had decent quality but the latency was a real drag.
Gemma MoE — okay. Nothing wrong with it, but Qwen models were consistently higher quality. Gemma felt like it was trying its best but falling short on the harder problems.

TL;DR

Qwen Coder Next is the best. Qwen 35B MoE is the best on balance. Cohere North Mini is extremely fast but unreliable on actual problem-solving. Gemma MoE is fine but Qwen wins on quality. Qwen 27B needs another shot.

On Syntax-Guided Program Reduction

Mon, 01 Jun 2026 00:00:00 +0000

Program reduction is an interesting problem: given an objective (e.g., a unit test), can we automatically create the minimal viable program by deleting unnecessary code?

This repo (nappe) and PyPI package is an implementation of an existing approach with extensions to solve this. As this problem is NP-hard, it’s not surprising that it is a tad ’expensive’ to do this well. However, it is interesting as it tackles things like removing dead code which perhaps tools like ruff don’t succeed with.

Ralph Loop and Frequent Intentional Context Compaction

Sun, 01 Feb 2026 00:00:00 +0000

Ralph Loop and Frequent Intentional Context Compaction

February 2026

There’s been a few things I’ve been playing around and thinking about, specifically around how one might ‘implement’ the ideas.

Beads
Ralph Wiggum Loop
Context Compaction and Management

To that end, I have a single Python script example.

This aims to address a few items:

Manage cross tasks through the use of ’tickets’ (similar to beads), but in a more prescriptive way, where the tickets are created and actively closed outside of the agent loop (n.b. the agent can also create tickets, this way is more intentional which I’ve found works better for a ‘human’ reviewer, cause then the determinism guarentees that the ticket is seen and reviewable.
The Ralph Wiggum loop manages the completion by assessing the ticket queue and seeing whether or not it is complete or not
As part of injecting context to the Ralph Wiggum loop (since context is not preserved), we make use of Context Compaction workflow which is the ‘Research, Plan, Implement’ steps, whereby explicit guidance for researching, then planning is made to populate the context before finally implementing

In general this loop works well, particularly for more complex tasks. The Ralph loop guarentees that it will at least see the task to completion.

Thinking Local LLMs and AI

Mon, 01 Dec 2025 00:00:00 +0000

Thinking Local LLMs and AI

December 2025

Running models locally is nothing new. Infact I’ve always had a particular affinity to llama.cpp. Recently, there is the newly introduced local text to image (z-image-turbo) generation model that can ‘comfortable’ be run locally (albeit perhaps a bit slow without a dedicated GPU).

Usage would look something like (using justfile to template it) using stable-diffusion.cpp:

 1[no-cd]
 2sd_generate:
 3 PROMPT=$(gum input --placeholder "prompt for image generation"); \
 4 OUTPUT=$(gum input --placeholder "output png file"); \
 5 DYLD_LIBRARY_PATH=/path/to/dyld/library \
 6 sd --difffusion-model z_image_turbo-Q4_0.gguf \
 7 --vae /path/diffusion_pytorch_model.safetensors \
 8 --llm Qwen3-4B-Instruct-2507-Q6_K.gguf \
 9 --cfg-scale 1.0 \
10 --offload-to-cpu \
11 --diffusion-fa \
12 -H 512 -W 512 --steps 9 \
13 -p "$PROMPT" \
14 -o "$OUTPUT"

On M1 Macbook Pro with offload cpu enabled it will take roughly 2 minutes per a step, whereas not offloading will improve performance at the cost of memory consumption (n.b. you should have --offload-to-cpu turned on if you are using a low memory variant).

Examining Pokemon AI as Inspiration for Scoring Systems

Sat, 01 Nov 2025 00:00:00 +0000

Examining Pokemon AI as Inspiration for Scoring Systems

November 2025

With LLMs all the rage, there is a desire for it to make more informed decisions. Perhaps there are multiple (good) choices one can make – how would we induce an LLM to select a ‘good’ choice?

One way to do this is using LLM as a judge. This boils down to a score card style system. The loop looks like this:

Using Outlines for LLM Constrained Generation

Wed, 01 Oct 2025 00:00:00 +0000

Using Outlines for LLM Constrained Generation

October 2025

Constrained generation is something that has interested me recently. Mostly as an extension of structured generation. For example in the newest gpt-5 models you can now have Regex as a constrained output. Now outlines is not particularly new, though what is interesting to me is the design of their APIs.

Chat Templates

Are created via:

1# Fill in nested templates
2prompt = chat_template(
3 system=system_template(instruction="You are a helpful assistant."),
4 user=user_template(query="What is machine learning?")
5)

Constrained outputs

What if I tried self-hosting LLM Code Tooling?

Tue, 01 Jul 2025 00:00:00 +0000

What if I tried self-hosting LLM Code Tooling?

July 2025

Currently Code AI tooling is going through a weird cycle. We have the Windsurf x OpenAI deal falling through, Cursor with some interesting (bad) pricing changes, Claude Code being a loss leader – which leads me to think, so what would self-hosting this on a laptop look like?

Firstly, there are a lot of options which exist already, however the reality is that my using of AI auto-complete and agentic mode is fairly limited. With that in mind, I think my current setup is rooted mainly in the llama.cpp world of things.

LLMs - in Review (2024)

Sun, 01 Dec 2024 00:00:00 +0000

LLMs - in Review (2024)

December 2024

2024 was the first year where I took LLMs seriously. I successfully hosted a Llama 70b parameter model in production which was used as with continue.dev for a self-hosted co-pilot replacement, along with a code autocomplete like Qwen Coder or Deepseek, these were fine replacements and surprisingly robust. Huggingface’s TGI along with Triton Server were the main heroes for this project, (Triton was used to serve onnx models for embeddings) though I’ve yet to find a “good” embedding model. At this stage in time, most of the vector database solutions “feel” the same and can all seemingly be trivially hosted via Kubernetes.