ML on NoRaincheck

Inpainting with Generative AI

Mon, 01 Jun 2026 00:00:00 +0000

Inpainting with Generative AI

If anything, trying to do generative AI (images) via CLI is unusually ‘hard’. Mostly because most flows use ComfyUI. I have found ComfyUI to be great when trying things out, or doing things interactively.

The easiest way to use CLI/scripting has definitely been stable-diffusion.cpp: https://github.com/leejet/stable-diffusion.cpp

For inpainting, it looks like the below

 1./bin/sd-cli \
 2 --diffusion-model flux-2-klein-9b-Q4_0.gguf \
 3 --vae flux2_dev_diffusion_pytorch_model.safetensors \
 4 --llm Qwen3-8B-Q3_K_M.gguf \
 5 --init-img bench.jpg \
 6 --mask dog-bench-mask.png \
 7 -p "a lovely dog" \
 8 --cfg-scale 2 \
 9 --sampling-method euler \
10 -t 24 \
11 --color \
12 --steps 9 \
13 -H 512 \
14 -W 512 \
15 --vae-tiling \
16 --vae-tile-overlap 0.125 \
17 -o dog-lovely-bench.png

What is important (to me) is that I find that binary masks that ‘grows a bit’ is better than providing a fuzzy mask, on my hardware this runs reasonably quickly (~15s per step at 512x512).

Maya1 vs Kokoro vs Kitten TTS Review

Sun, 01 Mar 2026 00:00:00 +0000

Maya1 vs Kokoro vs Kitten TTS Review

March 2026

I finally got to test TTS models, and got them running locally with a variety of notes.

Kitten: advertised as the smallest model, its also the easiest to setup
Kokoro: an extremely good model for its size. I found getting it setup with the onnx wrapping to be the most straightforward (supports quants!)
Maya1: supports gguf. I ended up hosting it in LM Studio with a wrapper to have it working. This is the best quality but also the slowest

TLDR: Use Kokoro if you want a good balance of speed and quality, otherwise Maya1 is a suitable model if you’re willing to wait a bit

Thoughts of ffmpeg and whisper filters

Thu, 01 Jan 2026 00:00:00 +0000

Thoughts of ffmpeg and whisper filters

January 2026

I’ve been experimenting with ffmpeg and the whisper filters. In general I think its awesome that such functionality exists, but at the same time, I don’t believe it addresses the particular painpoints when you go beyond the ‘obvious’ thing.

Installation

On macos to install ffmpeg with the whisper filters, the easiest way is via brew:

brew tap homebrew-ffmpeg/ffmpeg
brew install homebrew-ffmpeg/ffmpeg/ffmpeg --with-whisper-cpp

Considerations

One of the cool functionalities of whisper-cpp is the ability to integrate voice activity detection (VAD). This works out of the box with the whisper filter. Unfortunately what does not work is integrating translations, instead it is expected you create the translation yourself.

Faster AutoML Random Search

Mon, 01 Dec 2025 00:00:00 +0000

Faster AutoML Random Search

December 2025

In today’s modern world of ML it is increasingly uncommon to perform full cross-validation when tuning models. Instead a lot of the focus (particular in the DL space) is to make use of train/validation split with a separate holdout with ablations.

Based on this trend, I believe when training non-DL models, we should employ the same approach.

When doing benchmarking for this, it ends up ranging from a 2.4x to 27.8x speed improvement for doing hyperparameter tuning.

Thinking Local LLMs and AI

Mon, 01 Dec 2025 00:00:00 +0000

Thinking Local LLMs and AI

December 2025

Running models locally is nothing new. Infact I’ve always had a particular affinity to llama.cpp. Recently, there is the newly introduced local text to image (z-image-turbo) generation model that can ‘comfortable’ be run locally (albeit perhaps a bit slow without a dedicated GPU).

Usage would look something like (using justfile to template it) using stable-diffusion.cpp:

 1[no-cd]
 2sd_generate:
 3 PROMPT=$(gum input --placeholder "prompt for image generation"); \
 4 OUTPUT=$(gum input --placeholder "output png file"); \
 5 DYLD_LIBRARY_PATH=/path/to/dyld/library \
 6 sd --difffusion-model z_image_turbo-Q4_0.gguf \
 7 --vae /path/diffusion_pytorch_model.safetensors \
 8 --llm Qwen3-4B-Instruct-2507-Q6_K.gguf \
 9 --cfg-scale 1.0 \
10 --offload-to-cpu \
11 --diffusion-fa \
12 -H 512 -W 512 --steps 9 \
13 -p "$PROMPT" \
14 -o "$OUTPUT"

On M1 Macbook Pro with offload cpu enabled it will take roughly 2 minutes per a step, whereas not offloading will improve performance at the cost of memory consumption (n.b. you should have --offload-to-cpu turned on if you are using a low memory variant).

Embedding Alignment

Fri, 01 Aug 2025 00:00:00 +0000

Embedding Alignment

August 2025

Why?

Given the increasing use of vendored vector store solutions and dependencies on AI for mission critical systems, it is important to determine ways to ensure systems stay reliable when third party APIs degrade. Embedding alignment can be an approach to re-map embeddings from one vendored system to another in the hope that systems stay reliable at the cost of minimal degradation at inference time. This approach requires only maintaining a linear mapping between embeddings rather than duplicating vector stores for multiple vendors which can be expensive in terms of ownership, processes and infrastructure.

What if I tried self-hosting LLM Code Tooling?

Tue, 01 Jul 2025 00:00:00 +0000

What if I tried self-hosting LLM Code Tooling?

July 2025

Currently Code AI tooling is going through a weird cycle. We have the Windsurf x OpenAI deal falling through, Cursor with some interesting (bad) pricing changes, Claude Code being a loss leader – which leads me to think, so what would self-hosting this on a laptop look like?

Firstly, there are a lot of options which exist already, however the reality is that my using of AI auto-complete and agentic mode is fairly limited. With that in mind, I think my current setup is rooted mainly in the llama.cpp world of things.

Setting SQL for Feature Transformations as a Standard

Sat, 01 Mar 2025 00:00:00 +0000

Setting SQL for Feature Transformations as a Standard

March 2025

So Spark’s SQLTransformer is probably the first (and only?) documented, formal, specification for doing SQL transformations specifically for machine learning preprocessing. I think that is interesting for a variety of reasons, with the biggest one being opportunities for standardisation, in particular for real-time flows. Having a well-optimised SQL transformation engine could be immensely valuation. The reason why this hasn’t occured is probably because industry standard today still relies on Python as the execution engine, however this because untenable in scenarios where Python is an inappropriate production programming language choice. At the same time, Spark is typically too expensive of a dependency to justify low latency workflows. Nevertheless lets quickly look at the specification and considerations for using the language with an alternative computation backend.

LLMs - in Review (2024)

Sun, 01 Dec 2024 00:00:00 +0000

LLMs - in Review (2024)

December 2024

2024 was the first year where I took LLMs seriously. I successfully hosted a Llama 70b parameter model in production which was used as with continue.dev for a self-hosted co-pilot replacement, along with a code autocomplete like Qwen Coder or Deepseek, these were fine replacements and surprisingly robust. Huggingface’s TGI along with Triton Server were the main heroes for this project, (Triton was used to serve onnx models for embeddings) though I’ve yet to find a “good” embedding model. At this stage in time, most of the vector database solutions “feel” the same and can all seemingly be trivially hosted via Kubernetes.

Python & TypeScript - in Review (2024)

Sun, 01 Dec 2024 00:00:00 +0000

Python & TypeScript - in Review (2024)

December 2024

One thing that I like to stress is the importance of tooling and relying on defaults. By being able to speak consistently within ones own projects or using commonly seen patterns reduces the mental overhead. These could be folder structures or idioms, especially things which permeate across different programming languages or frameworks.

Here are some of my thoughts on Python and TypeScript; coming from someone who is predominantly a Python developer and does minimal front-end work.