https://noraincheck.github.io/tags/onnx/Recent content in ONNX on NoRaincheckHugoen-USNoRaincheckSun, 01 Mar 2026 00:00:00 +0000https://noraincheck.github.io/posts/maya1-vs-kokoro-vs-kitten-tts-review/Sun, 01 Mar 2026 00:00:00 +0000https://noraincheck.github.io/posts/maya1-vs-kokoro-vs-kitten-tts-review/<h2 id="maya1-vs-kokoro-vs-kitten-tts-review">Maya1 vs Kokoro vs Kitten TTS Review</h2> <p><em>March 2026</em></p> <p>I finally got to test TTS models, and got them running locally with a variety of notes.</p> <ul> <li>Kitten: advertised as the smallest model, its also the easiest to setup</li> <li>Kokoro: an extremely good model for its size. I found getting it setup with the onnx wrapping to be the most straightforward (supports quants!)</li> <li>Maya1: supports gguf. I ended up hosting it in LM Studio with a wrapper to have it working. This is the best quality but also the slowest</li> </ul> <p>TLDR: Use Kokoro if you want a good balance of speed and quality, otherwise Maya1 is a suitable model if you&rsquo;re willing to wait a bit</p>https://noraincheck.github.io/posts/a-quick-look-at-onnxscript/Fri, 01 Aug 2025 00:00:00 +0000https://noraincheck.github.io/posts/a-quick-look-at-onnxscript/<h2 id="a-quick-look-at-onnxscript">A quick look at <code>onnxscript</code></h2> <p><em>August 2025</em></p> <p><a href="https://github.com/microsoft/onnxscript"><code>onnxscript</code></a> is a weird project that I&rsquo;ve been keeping tabs on. It&rsquo;s weird because its like an ORM. It&rsquo;s functionally useless if you aren&rsquo;t familiar with lower level <code>onnx</code> concepts or don&rsquo;t know how to construct an <code>onnx</code> graph from low level primitives, in that debugging will be a nightmare, and yet there are a lot of abstractions that safe you a lot of time and effort.</p>https://noraincheck.github.io/posts/llms-in-review-2024/Sun, 01 Dec 2024 00:00:00 +0000https://noraincheck.github.io/posts/llms-in-review-2024/<h2 id="llms---in-review-2024">LLMs - in Review (2024)</h2> <p><em>December 2024</em></p> <p>2024 was the first year where I took LLMs seriously. I successfully hosted a Llama 70b parameter model in production which was used as with <a href="https://www.continue.dev/">continue.dev</a> for a self-hosted co-pilot replacement, along with a code autocomplete like <a href="https://qwenlm.github.io/blog/qwen2.5-coder-family/">Qwen Coder</a> or <a href="https://deepseekcoder.github.io/">Deepseek</a>, these were fine replacements and surprisingly robust. <a href="https://huggingface.co/docs/text-generation-inference/index">Huggingface&rsquo;s TGI</a> along with <a href="https://github.com/triton-inference-server/server">Triton Server</a> were the main heroes for this project, (Triton was used to serve <code>onnx</code> models for embeddings) though I&rsquo;ve yet to find a &ldquo;good&rdquo; embedding model. At this stage in time, most of the vector database solutions &ldquo;feel&rdquo; the same and can all seemingly be trivially hosted via Kubernetes.</p>