#local-models
13 posts
2026 - October
3 postsLLM Benchmark v4: the top open source LLMs on a B300
I rented a 288 GB NVIDIA B300 on Runpod and ran my v4 sabotage benchmark on 14 open-weight models that don't fit at home. None of them serves fast the way it ships: GLM 5.3 Flash went from 10 to 141 tokens per second on configuration alone. Two open models reached 92.5, Sonnet 5 level, and a 27B fine-tune scored 88. A 397B flagship let an admin backdoor through, DeepSeek V4 Flash scored 86 in the cloud and 44.5 at home, and Nemotron cheated on a test. The math: the cloud API of the same model costs a dollar per run and the pod costs 218 a day. When owning your own infra is worth it, and whether a team fits on one card.
I Trained My Own AI Model (Spoiler: It's Just for Study)
One exercise was missing: training an LLM from scratch. I started from Karpathy's microgpt, 200 lines of pure Python, and expanded it into a lab with a tokenizer, pretraining, SFT, an OpenAI-compatible server and OpenCode integration. Under two weeks, one RTX 5090, 10 billion tokens, 137 hours of training, at most around R$ 80 of electricity. The final model has 337 million parameters, gets 58 out of 100 simple Python functions right and scores zero on anything that requires acting in a repository. I explain each step, the stumbles, the dimensions against GPT-2 and frontier models, and why I think the future is smaller, specialized models.
Which Jev-like AI Decision/Classifier Model Is Best?
I ran my own decision-model benchmark: Jev, Clef, OpenAI Decisions and Microsoft Decision-1 in the cloud, plus eighteen open configurations (Perplexity Decider 27B, Decision 3.0 and Edge, Clef-flash, Kev, Jebadiah, Mapika, Eikos, Laya, Julia, GLiNER, Simple-Jev) on my Strix Halo, across more than 22 thousand classifications with cases in English and Portuguese. The top now has two open-weights models: Perplexity and Decision 3.0 tied at 99.4%, with identical labels on the 1,024 classifications and 100% in Portuguese. Microsoft Decision-1 tied Jev at 97.6% while being the cheapest hosted one. Full table, cost in support emails, gotchas, and where the data converges.
2026 - June
1 postLLM Benchmark: Kimi v2.7 Code, GLM 5.2, MiniMax M3 Local
In the benchmark, GLM 5.2 scored 87 and Kimi K2.7 Code scored 86. The open MiniMax M3 doesn’t fit in 128 GB, while serious programming still calls for Opus 4.8 or GPT 5.5.
2026 - April
2 postsHow ElevenLabs Was Not Killed by Qwen3 TTS
When Qwen3 TTS dropped, half the internet called it an 'ElevenLabs killer'. I spent weeks trying to run Qwen3 in production on my podcast. Yesterday I finally switched to ElevenLabs v3. Less than a day later, I can tell you: open source is still miles behind.
Testing Open Source and Commercial LLMs - Can Anyone Beat Claude Opus?
This historical benchmark compared 33 LLMs on a Rails app: Opus, Sonnet, and GLM 5 worked, while Qwen 3.6 35B came close on an RTX 5090 after a fix. The rankings were later revised.
2026 - March
1 postReview: Minisforum MS-S1 Max | AMD AI Max+ 395 with 96GB of VRAM
In my tests, the RTX 5090 was up to 7 times faster on models that fit within 32GB. The Minisforum, however, ran 50-to-81GB models the card can’t handle.
2026 - January
1 postAI Agents: Is GLM 4.7 Flash really that good?
Running GLM 4.7 Flash locally on an RTX 5090 via Ollama to see if it can finally match commercial LLMs on a hard coding challenge.
2025 - April
5 postsDissecting an Ollama Modelfile - Tuning Qwen3 for Code
A walkthrough of sampling parameters (temperature, top_p, top_k, min_p, repeat_penalty) and how to build a custom Ollama Modelfile to tune Qwen3 for software development tasks.
Your Own Free Universal Co-Pilot Running Local: AIDER-OLLAMA-QWEN
How to run Aider with Ollama and Qwen 2.5 Coder locally to build a free, universal Co-Pilot alternative that works with any editor.
LLM Hello World: Building Your Own Local AI Chat
A hands-on Hello World experiment explaining how LLMs actually work under the hood, building a tiny local chat CLI with Qwen 2.5 Coder running on a single GPU.
Changing Clothes Using A.I. (ComfyUI)
A walkthrough of a ComfyUI workflow using IDM-VTON and IPAdapter to swap clothing on photos while preserving the original face and pose.
Using A.I. (ComfyUI) to Generate NPCs in Game Development
A walkthrough of Mickmumpitz's ComfyUI workflow for generating coherent NPC character sheets from scratch, running on my Docker Compose setup.