I got into the ancient game of Go because of the manga Hikaru no Go. I even wrote about it back in 2008, in a post that’s still up, arguing why programmers should play Go. Since then, my use case has barely moved: I own a physical goban at home, but I never had the slightest intention of joining a study group, a federation, or a tournament. I just like playing solo, solving tsumego (Go’s life-and-death puzzles), and loading famous games to study, like the legendary Lee Sedol vs AlphaGo games from 2016.

For anyone who doesn’t know the manga: Hikaru no Go (written by Yumi Hotta, drawn by Takeshi Obata, the same artist as Death Note) ran in Shonen Jump from 1998 to 2003 and set off a Go boom across all of Japan. And here’s a detail few people know: practically every game shown in the manga is based on a real historical game, recreated panel by panel. Hold onto that, because it comes back in a minute.

SGF and the scattered-material problem

There are several very good open source programs for loading Go games or even playing online. Go games are recorded in a format called SGF (Smart Game Format), a plain-text format that’s been around since the late 80s: each move becomes a coordinate (;B[pd] is a black stone on the 4-4 point), and the format supports variation trees, comments, and game metadata. It’s the de facto standard of the Go world: every client reads SGF, every server exports SGF, and there’s a gigantic archive of historical games and problems in this format accumulated over decades.

And that’s where the problem lives: that archive is scattered across the entire internet. One site has the classic tsumego collections, another has the historical games, one GitHub repo has the commented problems, another has the professional kifu (game records). I was in no mood to download everything by hand, organize it folder by folder, and configure client by client. I wanted an app good enough, pretty enough, with everything built in.

2008: the GNU Go era

Here’s where the historical context earns its keep, because it’s the fun part. In 2008, when I wrote that post, the only accessible Go “AI” was GNU Go. And look, you couldn’t even call that thing an AI: it was a pile of stacked heuristics. And it was terrible. Even an amateur like me could see it made moves with no flow, no sense of direction, loose stones that didn’t talk to each other. It beat beginners on pure local tactics and nothing else.

What happened over the next 18 years is one of the most beautiful stories in computing. First came the Monte Carlo Tree Search methods, which already raised the level considerably.

Then in March 2016, DeepMind’s AlphaGo beat Lee Sedol, one of the greatest players in history, 4 to 1. That was the series of the famous move 37 in game 2 (the shoulder hit no human would play) and Lee Sedol’s answer in kind: move 78 in game 4, the “hand of God” that earned him his only win. If you’ve never seen it, there’s an excellent documentary about that series on Netflix, worth watching even if you know nothing about Go.

And after DeepMind published the papers, the open source community chased it down: first Leela Zero reimplementing AlphaZero, then KataGo going beyond it. Today anyone has a superhuman Go player running for free on their own computer. I wanted to see up close how much these AIs matured in 20 years.

That itch is what my new toy project was born from: Frank GO.

Sabaki + KataGo

I asked Claude Code to research the most popular open source Go apps and engines (the whole survey is documented in the repository itself, as it should be). In the end I picked two projects as the foundation:

Sabaki, by Yichuan Shen, is the front-end. It’s probably the most beautiful, best-kept open source SGF editor and Go board out there, built on Electron, with variation trees, analysis, and engine support via the GTP protocol. Elegant and minimalist.

KataGo, by David Wu, is the AI. It’s the strongest open source engine in the world, superhuman, heir to the AlphaZero lineage but with one difference that matters enormously for my case: beyond win rate, KataGo estimates score and territory ownership point by point. It answers “who’s winning, by how many points, and which areas of the board belong to whom.” That’s exactly the kind of information a beginner needs and that the old engines kept hidden.

Frank GO is a fork of Sabaki that wires in KataGo for solo play sessions, and bundles everything I wanted to have in one place:

• Over 4,700 tsumego from the classic collections (Cho Chikun’s Encyclopedia, the Gokyo Shumyo, the Xuanxuan Qijing), transcribed by Vít Brunner at tsumego.tasuki.org and converted to SGF by the tasuki2sgf project, plus the 420 commented problems from Go Game Guru’s go-problems (by An Younggil, 8-dan professional), organized into 10 difficulty levels. Solve 5 in a row and you level up. A focus picker filters by life & death, tesuji, ko, or semeai (capturing races).
• The puzzles fight back: with KataGo installed, it answers your moves inside the problem, and when it gives up the area the app judges the result and marks the problem solved automatically.
• Famous historical games to study, from the Ear-Reddening Game (1846, by the legendary Honinbo Shusaku) to AlphaGo vs Lee Sedol, curated from Andries Brouwer’s game database at CWI, each with the story behind it, and the AlphaGo ones with move-by-move commentary. You can watch them in auto-play or turn on “guess the moves” and try to find the pro’s play.
• The Hikaru no Go games: remember how the manga’s games are real? Well, 15 of them are bundled with chapter references and trivia, using the chapter-to-game mapping the community compiled on Sensei’s Library. Sai’s internet game against Toya Meijin, for instance, is a real game decided by half a point.
• A joseki dictionary (Kogo’s Joseki Dictionary, the classic corner-opening reference), ladder drills, a “who is winning?” drill, and a ten-problem rank test that estimates your level and drops you straight into the right practice.

Remember the scattered-material problem? Those collections above are exactly it: each one lived in its own corner of the internet. The provenance and licensing of every set is documented in the repo’s data/SOURCES.md, because you use other people’s archives by giving proper credit.

Overlays for people just starting out

The part of Sabaki I changed the most was building beginner-friendly visual overlays. The main one is area painting: an overlay that paints each player’s influence as a soft gradient and settled territory in solid color, so you literally see what the stones are doing, which areas are dead and which are contested.

Alongside it comes the real-time score accounting for komi (White’s point compensation for playing second), which warns you when you’re hopelessly behind or when the game is already settled. And hovering over the board shows the move names (“Hane”, “One-Point Jump”), the vocabulary the books use and that beginners take forever to absorb.

That was my problem with every Go software I ever tried: they’re tools built by strong players for strong players. A winrate percentage says nothing to someone who’s learning. “You’re 12 points behind and the lower left corner just died” says everything. KataGo always knew how to compute this; someone just had to put it on screen in a way a beginner understands.

The idea is to be a one-stop shop, zero configuration, for beginners and students to learn Go at their own pace. And there’s a build for everywhere. On Arch Linux it’s yay -S frank-go via AUR. On Windows there’s an installer and a portable build on the Releases page; the installer has no code signing, so SmartScreen complains on first run — just click “More info → Run anyway” (I explain why I don’t sign Windows builds in the best-practices post). On macOS it’s brew install --cask akitaonrails/tap/frank-go, or the .dmg for Apple Silicon and Intel; that build is signed with a Developer ID and notarized by Apple, opens with zero drama. And on any platform you can run from source with npm install.

In every case the app downloads a small CPU engine with one click, no GPU, no account, no server, fully offline. The exception is macOS, where KataGo has no official binary: a brew install katago unlocks play-vs-AI, and everything else (tsumego, study, drills, area painting) works without it. Your progress stays on your machine.

And when you want to play against people

Frank GO is deliberately a solo study app. When you’ve learned and want to play against humans, the natural path is the online servers, and there the more advanced apps serve you better. OGS (Online Go Server) is the friendliest today, runs right in the browser. KGS and Pandanet IGS (the original Internet Go Server, running since the early 90s) are the veterans, and the Asian servers like Fox and Tygem concentrate the strongest players. Organized Go revolves around the IGF (International Go Federation) and the national associations, with rankings and tournaments for anyone who wants to take it seriously.

I don’t. And that’s the whole point of the project: like I said up top, I have zero desire to compete. What I wanted, for literally years, was a way to play and have fun with Go at my own pace, with decent study material in one place, against an AI that plays beautifully instead of spitting out senseless stones like the GNU Go of 2008. I think I finally pulled it off.

The code is MIT, the repo is github.com/akitaonrails/frank_go, and contributions are welcome. There’s even an open issue asking for character portraits for the Hikaru no Go mode, if anyone feels like drawing. Sai thanks you.

I’m a subscriber to the MANGA Plus by Shueisha app on Android’s Google Play Store. For those who don’t know, this is Shueisha’s official channel (the Japanese publisher behind Shonen Jump) for reading their manga legally, with chapters releasing weekly nearly the same day they’re published in Japan. Two dollars a month gets you the entire catalog with an official English translation.

I own a Samsung ZFold 7, one of the few phones that’s actually good for reading manga and Kindle (I talked about this and about Frank Yomik in the AI marathon final ranking). The Fold opened gives you a bigger screen than a normal phone, which helps a lot. But there are days when my eyes are already tired of tiny screens, and what I really wanted was to read on a 32-inch OLED monitor that sits right in front of me, perfect for long sessions.

The problem is that MANGA Plus has no official website for reading. The subscription is locked to Android. There’s an iOS version too, but no web. A tablet helps, but nothing beats the comfort of a big, static screen when you want to read a whole One Piece arc in one go.

For context, I caught up on a lot recently: I finally finished Chainsaw Man, Dan Da Dan, Jujutsu Kaisen, Kagurabachi, Akane-banashi, Sakamoto Days and, of course, One Piece. There’s a lot more I want to follow, and the comfort of the big monitor became a real problem worth solving.

The main act here is Frank Manga+, which fixes Shueisha’s desktop gap. At the end I also cover two pieces of my setup: Prettify Manga Reader, for turning fansub sites and Kindle Web into tolerable readers, and Frank Yomik in the browser, for furigana/translation on Kindle Japan and Naver.

Decompiling the app

I decided to dig in. I sat down with Claude Code and we downloaded the official MANGA Plus xapk. We decompiled it to Java source with jadx and navigated around until we understood how the app authenticates with Shueisha’s API.

The conclusion is straightforward: the user’s “subscription,” from the app’s point of view, is a 32-character hexadecimal deviceSecret stored in a plain XML file inside Android’s shared preferences. The app generates this secret once when you log in, and from then on every API call (jumpg-api.tokyo-cdn.com) is signed with it. No elaborate DRM, no rotating token, none of that. A single static secret per device.

The catch is that extracting that secret from the app is a hassle, because by default you don’t have access to /data/data/jp.co.shueisha.mangaplus/shared_prefs/config.xml on a non-rooted Android. Three paths exist:

1. Rooted phone: run adb shell su -c "cat /data/data/jp.co.shueisha.mangaplus/shared_prefs/config.xml" and look for <string name="secret">...</string>.
2. Rooted emulator: install Android Studio, create an AVD with Google Play x86_64, apply rootAVD for Magisk root, log into the Google account that holds your subscription, install MANGA Plus from the Play Store, then run the same adb shell above.
3. Network capture with mitmproxy (more advanced): run the rooted emulator, inject the mitmproxy CA via Conscrypt APEX, and intercept the secret on the app’s first authenticated call.

All three paths are documented in detail in the repo’s docs/install.md and docs/debugging.md. I won’t repeat them step by step here because it’s more about tooling than technique, and anyone willing to put in the work will want to read the original. The important thing is that it works, and you only have to do it once: once extracted, the secret keeps the desktop app working for as long as your subscription lasts.

Yes, it’s annoying. But it’s the price for having a single configuration point that unlocks the rest.

The desktop app

With the secret in hand, the rest was easy. I built Frank Manga+, a native app in Rust + Tauri, with a SvelteKit WebView on the frontend and a Rust client using reqwest on the backend talking directly to Shueisha’s API. To load chapter images without leaking URLs or fighting CORS, I created a custom mpimg:// scheme intercepted by Tauri, which downloads, decrypts (the API serves pages with a simple XOR), and hands the result back to the WebView.

The app opens on a library view with search:

Clicking a title takes you to a detail page with the full chapter list and a “Continue” button that picks up from the last chapter you read:

The reader opens in single-page, with snap-per-page and manga-style click zones: click the left half of the page to advance and the right half to go back (Japanese right-to-left reading direction). If you’d rather use the keyboard, arrows, Space, j/k, and PageUp/PageDown all work. When you hit the end of a chapter, the next one prefetches and gets appended to the scroll, so you don’t have to bounce back to the chapter list every time. And resume is per-chapter: leave mid-read and the next time you open that chapter you land on the exact page you stopped at. For huge series like One Piece (1100+ chapters), the list is virtualized, so it opens instantly:

On a 32-inch monitor, single-page wastes half the screen. That’s why there’s double-page mode: hit D (or click the layout icon in the header) and the reader starts pairing facing pages, so the whole spread fills the screen the way the author drew it. It’s how manga was meant to be read.

You can see why that matters when you compare it with cheap binding. Bad paperbacks swallow the middle of the fold, exactly where the art is supposed to connect. In digital, when the reader joins it properly, the scene gets room to breathe:

Frank Manga+ joining the spread	Cheap paperback swallowing the fold

D cycles through three layouts: single (one page at a time), double (sequential pairs starting from page 1), and double-cover (the first page of each chapter solo, the rest in pairs, the way printed manga binds the cover singly before the first spread). The choice persists across chapters and sessions, so you set it once and forget it.

There’s also a night-reading filter. Hit F (or click the crescent-moon icon in the header) and the page whites warm toward sepia. LCD white is harsh on the eyes in the dark, and this softens it without flattening the contrast:

F cycles through four levels: off → low → med → high → off. The button tints amber and shows one to three dots for the active level, and the choice is saved in localStorage like the reader’s other prefs. Under the hood it’s a CSS sepia + brightness + saturate filter on the whole page-stack: the sepia pushes the hue toward amber but preserves the luminance range, so blacks stay black and the artwork’s contrast is intact.

Reading state (last chapter, current page) lives in the WebView’s localStorage. Image cache at ~/.cache/mangaplus-reader/. The extracted deviceSecret lives in ~/.config/mangaplus-reader/secret (Linux/macOS) or %APPDATA%\mangaplus-reader\secret (Windows). You can override it with the MANGAPLUS_SECRET environment variable if you prefer.

Install

There’s a build ready for all the main platforms:

• Linux: AppImage, .deb, .rpm, or via AUR (yay -S mangaplus-reader-bin).
• macOS: separate .dmg for Apple Silicon (aarch64) and Intel (x64).
• Windows: .exe installer. It’s not signed, so Windows SmartScreen will complain, click “More info → Run anyway”.

All artifacts come out on the GitHub Releases.

Status and contributions

This is beta. It works fine for my use (I read several series a week with no headaches), but it’s missing features: genre filters in search, real offline mode (today it only caches what you’ve already read), and a smoother secret-extraction path for less technical users.

If you subscribe to MANGA Plus and want to test it, send an issue or PR. The code is Rust + Svelte, MIT, contributions welcome: github.com/akitaonrails/frank_mangaplus.

Now I finally read One Piece on the big monitor, without having to hold a phone or twist my neck. The work paid off.

Bonus: Prettify Manga Reader for fansub sites

Besides the desktop app, I keep a simpler extension around: Prettify Manga Reader. It does not translate anything. Its job is to fix the reading experience.

Most fansub sites are ugly, packed with banners, and obviously built for phones: one page under the next, endless vertical scroll, good luck. That’s practical on a phone, but manga was not drawn like an Instagram feed. Manga was drawn for page turns. Two pages open at once. The reading rhythm, the reveal, the pause between panels, all of it assumes that pair.

And many authors use that to draw two-page spreads. Kaiju No. 8 does it several times. If the site stacks one page under the other, you can still read the speech bubbles, but the full composition is gone. You’re seeing half the art at a time and thinking you saw the whole thing.

The usual site looks like this:

Prettify opens a dark overlay, fits the art to the screen, and joins pages at the center seam when Book or Double mode is active. The spread looks like a real two-page spread again:

It has three modes: Single (one page per screen), Double (pairs from the start), and Book (first page alone, then pairs, like binding). It opens in Book by default. If an image is already horizontal, it stays as a full-width spread. It has scroll snap so every page lands in the right place, keyboard shortcuts, small mouse controls, help with ?, and an end-of-chapter card that tries to find previous/next chapter links when the site makes them obvious.

It also has a three-level night filter. Hit N and it warms the page with sepia, brightness, and contrast without turning the art into dead gray. A white manga page on a big monitor at night is a flashlight to the face. This fixes it.

To install it, go to the Prettify Manga Reader GitHub repo, download the zip from Releases, unzip it into a permanent directory, open chrome://extensions or brave://extensions, enable Developer mode, click Load unpacked, and select the unzipped folder. If you want to build from source, clone the repo, run npm test, then npm run package, unzip the package generated under dist/, and load that folder.

On Kindle Web Reader, Prettify deliberately does less. On URLs like read.amazon.com/manga/..., read.amazon.co.jp/manga/..., read.amazon.co.uk/manga/..., and read.kindle.* domains, it doesn’t try to rebuild the book inside its own overlay. Amazon’s reader already handles account, pages, and DRM; the extension only adds a discreet bottom-right toolbar and keyboard shortcuts on top of the native reader.

Space, PageDown, and arrows advance the page; Shift+Space, PageUp, and the opposite arrows go back; Home/End jump to the beginning or end when the native reader accepts them, with a scroll fallback; N cycles the night filter. It doesn’t store Amazon credentials, doesn’t touch sessions, and needs no token in the code. It’s just visual comfort and keyboard control on top of the Kindle Web that already exists.

Bonus 2: Frank Yomik now on the desktop too via extension

Anyone who read the AI marathon recap remembers Frank Yomik, my self-hosted project that takes a manga or webtoon page, detects the bubbles with RT-DETR-v2, runs OCR, sends it to a local Ollama (qwen3:14b) for translation, and returns the page with the translated text in place. I already covered the story and the technical details in My First Vibe Code Failure and How I Fixed It | Frank Yomik. The point here is the new desktop piece.

Originally it had only a Flutter client for Android and Linux, which opened Kindle Japan and Naver Webtoon inside a WebView. The thing is, running a Flutter app inside a WebView on the desktop just to read Kindle is using a sledgehammer to crack a nut. On the desktop you already have Chromium open. What was missing was a browser extension.

Over the past few weeks I added exactly that. A Manifest V3 extension for Chromium/Brave/Edge, talking to the same self-hosted Yomik server. It runs on read.amazon.co.jp, read.kindle.co.jp, comic.naver.com, and Naver’s mobile version. The page stays visually identical to the original, with no injected buttons or HUD in the reader’s face. All configuration lives in the popup:

The content script grabs the current page image and sends it to the server. The bearer token sits in the service worker, so it never leaks to the page’s JS. When the translation comes back, the image gets swapped in place of the original. There’s a local cache to avoid re-translating pages you’ve already seen, a “force reprocess” button for when you need to file a bug report, and autosave for the config. Details in the extension README, and the Load Unpacked zip ships with the repo’s Releases.

In practice, the 32" monitor is now my reading desk. Frank Manga+ opens Shueisha’s official catalog in English (One Piece, Chainsaw Man, and whatever else Shonen Jump publishes). When it’s fansub or Kindle Web and all I want is visual comfort, Prettify handles the reader, keyboard, and night filter. When I want to read something Japanese with no official translation, I open Kindle Japan in another tab and Yomik renders furigana or an LLM translation on top of the page. For Korean Naver webtoons, same story, same extension.

The nice part is that each project solves a different piece and they complement each other. Frank Manga+ exists because Shueisha has no website. Prettify makes browser readers less miserable. Yomik exists because the rest of the manga I read doesn’t have a decent official translation in any reasonable timeframe. For a manga addict like me, it’s the setup I’d been wanting for years.

I’ve always loved karaoke. I go out to sing with family or friends every now and then. In São Paulo there are good places in Liberdade and Bom Retiro, for instance, with private Japanese-style booths. If you’ve never been to a karaoke like that: you rent a private room by the hour, there’s a huge song catalog, two microphones, and a scoring system that grades your singing in real time. The best systems are Japanese, like Joysound and DAM. A score above 90 (out of 100) is considered advanced. DAM, in the LIVE DAM Ai series, even uses AI to give scores that feel more “human.”

But not every place has that level.

The problem with karaoke in Brazil

In Brazil we grew up with Videokê, the brand the Korean Seok Ha Hwang brought to the country in 1996, importing equipment from Korea. It became a craze in the 90s and 2000s, showed up in every bar, barbecue and birthday party. The problem is that those machines stopped in time. The current models, like the VSK 5.0, ship with around 12-13 thousand songs in the catalog, which you expand by buying cartridges or song packs. In practice, the repertoire is old, the interface is straight out of the 2000s, and if the song you want to sing came out after 2015, good luck.

The workaround a lot of bars adopted was to allow Chromecast or screen mirroring so that customers can search for songs directly on YouTube. Makes sense: on YouTube you can find karaoke for any song. With-lyrics version, instrumental version, vocal guide version.

But there’s a downgrade: you lose the scoring. One of the most fun parts of karaoke is the competition. Watching your score climb, comparing with friends, trying to beat the night’s record. If you’re just singing on top of a YouTube video, you get no feedback. It’s like bowling without a scoreboard.

And buying a professional system for home? Importing a Joysound F1 runs north of US$ 2,000 just for the hardware, not counting the monthly catalog subscription. For casual use it makes no sense.

The idea: YouTube with real-time scoring

Frank Karaoke came out of that frustration. If YouTube already has every song, why not build an app that works as a YouTube wrapper with a real-time scoring overlay? You search for any karaoke video, sing along, and the app analyzes your voice through the mic and shows a live score.

It’s a Flutter app for Android. Internally it loads YouTube into a webview and injects an overlay in HTML/CSS/JavaScript right into the page. The score display, the pitch trail, the settings panel, the mode selector — all of it rendered inside the webview through JS injection.

Scoring without a reference

Now, the real problem. Every professional karaoke system depends on prebuilt reference files for each song. Every single one.

Sony’s SingStar, which sold over 12 million copies between 2004 and the end of the PS3 era, had a hand-crafted note track for every song. Every note, every syllable, all mapped manually. The mechanism compared the singer’s pitch via FFT against that reference in real time. A detail I thought was clever: octave was ignored. If the right note was a C, it didn’t matter if you sang C3 or C4. Men sing women’s songs no problem.

Joysound and DAM in Japan go further and evaluate three separate dimensions: pitch accuracy (音感), rhythm/timing (リズム感) and expressiveness/dynamic volume (表現力). All based on MIDI data from the operator’s server. The open source equivalent format is UltraStar, where each song has a .txt file like:

: 12 4 5 Hel-    (NoteType StartBeat Duration Pitch Syllable)

Pitch 5 = MIDI 65 (F4). Scoring compares the singer’s pitch against the note’s pitch, modulo octave, with a tolerance of 1 semitone.

Frank Karaoke works with any YouTube video. There’s no reference file. There’s no MIDI. There’s no melody annotation. Zero metadata about what note you’re supposed to be singing.

I don’t know anything about karaoke scoring. I don’t know anything about audio processing, pitch detection, music theory applied to software. Nothing. So I asked Claude Code to do extensive research on the subject. What it brought back is documented in docs/scoring.md in the repository, and it’s a lot: academic papers on singing evaluation (Nakano et al. 2006, Tsai & Lee 2012, Molina et al. 2013), patents (Yamaha has one from 1999, US5889224A, that details MIDI-based scoring with 3 tolerance bands), and the source code of open source projects like UltraStar Deluxe, AllKaraoke, Vocaluxe and Nightingale.

The conclusion of the research: without a per-song reference, you have to evaluate vocal quality generically. Measure how the person is singing, not what they should be singing. And since no single metric works for every case, we decided to implement four different scoring modes, each measuring a different dimension of vocal quality.

The phone microphone problem

Before the scoring modes, I have to explain a more fundamental problem the research uncovered: the phone microphone.

When you sing karaoke with the phone, the mic picks up three things at once: your voice, the music coming out of the speaker, and ambient noise from the room. Your voice is physically closer to the mic, so it dominates the signal. But not enough for clean separation.

I tried several approaches to isolate the voice:

Spectral subtraction using YouTube’s reference audio. Dropped it. The YouTube CDN blocks direct audio extraction by non-browser user-agents, and even with the reference audio in hand, the speaker’s EQ, the room reverberation and the Bluetooth delay make the signal too different from what the mic captures. Naive subtraction produces artifacts worse than no subtraction at all.

Pre-emphasis + center clipping. Dropped that too. Center clipping destroys the waveform that the YIN algorithm needs for autocorrelation, and pre-emphasis amplifies noise as much as it amplifies voice.

What works is a 200-3500 Hz bandpass filter: a second-order IIR (Butterworth, Q=0.707) in cascade. The high-pass at 200 Hz kills bass, kick drum, bass guitar bleed from the speaker. The low-pass at 3500 Hz kills cymbals, hi-hats, high-frequency noise. Human voice fundamentals (85-300 Hz) and formants (300-3000 Hz) pass through the filter. It’s not perfect isolation, but it improves the voice/music ratio enough for pitch detection.

But the bandpass alone doesn’t solve everything. Guitars, synths and piano produce periodic signals in the same frequency range as voice, and YIN detects pitch in them too. To deal with that, the app does adaptive calibration: in the first 5 seconds of warmup (when nobody’s singing yet), it collects RMS samples from the signal to establish a baseline of the speaker’s level. During the song, it keeps that baseline updated (25th percentile of the last ~4 seconds of frames). For a frame to be scored, the RMS has to be at least 1.3x above the baseline. Your voice is closer to the mic, so it pushes the RMS above the speaker’s level. The instrumental melody stays near the baseline and gets filtered out. In testing, the original singer coming out of the speaker scored around 37 with sparse dots in the trail, while someone actually singing scored ~59 with dense dots.

Another annoying detail: on Android, specifically on Samsungs, the DSP’s AutomaticGainControl (AGC) attenuates the signal instead of amplifying it. On Galaxies, enabling AGC drops the mic peak from ~0.06 to ~0.003. Silence as far as pitch detection is concerned. So the app disables AGC, echo cancellation and noise suppression. When the peak falls below 0.01, it applies software gain (up to 30x) to bring the signal up to usable levels.

The YIN algorithm

To detect the voice’s pitch I use YIN, by Alain de Cheveigné (IRCAM-CNRS) and Hideki Kawahara (Wakayama University). It’s a fundamental frequency estimator in the time domain. The central idea is the Cumulative Mean Normalized Difference Function (CMNDF), which basically measures how periodic the signal is at each lag, normalizes it to reduce false positives, and uses parabolic interpolation to refine the result. It’s lightweight enough to run in real time on a phone, which is what matters here.

In the app, the YIN threshold is 0.70 (tuned for mixed voice + music signals), and frames with confidence below 0.3 get discarded. Below that, it’s probably noise or an instrument.

The 4 scoring modes

Each mode evaluates a different aspect of vocal quality. They all share the same audio pipeline (bandpass → YIN → confidence gate). The difference is how they interpret the detected pitch.

Pitch Match

Measures how cleanly you sustain notes. Uses Gaussian decay based on the standard deviation of MIDI values in a rolling ~15-frame window. Steady notes (deviation < 0.3 semitones) score 85-100%. A trembling voice (deviation > 2 semitones) scores near zero. Good for songs you already know well.

Contour

Measures the melodic shape of your singing. It doesn’t matter which exact note you hit, only the direction and the flow. Evaluates the pitch range and melodic movements (jumps > 0.5 semitone) in a rolling window. Monotone singing scores ~10%. Smooth melodic movement with a 2-6 semitone range scores 70-100%. Good for when you’re learning a new song.

Intervals

Measures the musical quality of jumps between consecutive notes. A whole tone (2 semitones) scores highest. Thirds and fourths score well. Wild jumps of an octave or more score low. Uses a Gaussian curve centered on the whole tone. Works when you’re singing in a different key from the original.

Streak

It’s Pitch Match with a combo multiplier. Each consecutive frame with a score above 0.4 increments the streak counter. The streak adds bonus points (up to +0.4 on a streak of 30+). Breaking a streak > 5 frames pushes a 0.05 penalty into the EMA. Silence freezes the streak, so instrumental breaks don’t hurt you. The most fun mode for parties.

The logic behind these four modes came from the research Claude did across academic papers. Each one measures a different dimension: pitch accuracy, melodic contour, phrasing and consistency. None of them is sufficient on its own, but together they cover, reasonably well, what you can evaluate without having the song’s reference melody.

The Pitch Oracle

Beyond the four purely vocal modes, the app has what I call the Pitch Oracle. The idea: instead of evaluating your voice in isolation, the app downloads the video’s reference audio via youtube_explode_dart, decodes it to PCM, runs YIN on it, and builds a timestamped pitch timeline of the entire song. During scoring, if the mic’s pitch matches the reference’s pitch at that moment in the video, it’s probably speaker bleed, and gets ignored. If it differs, it’s your voice, and gets scored.

The synchronization works through the currentTime of the HTML5 video element, sent to Dart through a JS timeupdate listener every ~250ms. The oracle queries the reference pitch at the exact playback position, accounting for pause, seek and speed change.

The first time you play a song, the oracle takes 5-15 seconds to download and analyze the audio. But the timeline is saved as JSON in the app’s local cache (pitch_oracle/<videoId>.json). If you play the same song again, it loads instantly from cache, no network request. That also fixes YouTube’s rate limiting problem for the songs you sing the most.

With the oracle active, the modes change behavior. Pitch Match compares the singer’s pitch class against the reference’s, agnostic to octave (like SingStar). Contour uses cross-correlation between the singer’s pitch movement and the reference’s. Intervals compares semitone jumps against the reference’s.

When YouTube blocks the download with rate limiting (happens after many consecutive requests from the same IP, clears in 15-30 minutes), the oracle silently fails and the modes fall back to purely vocal analysis.

The road to here

The app you see now went through a lot of iteration before reaching this state.

First, I tried to make a Linux desktop version to make debugging easier. Makes sense, right? Test on the desktop, iterate fast, then port to mobile. The problem is that Flutter has no webview backend for Linux desktop. webview_flutter simply doesn’t work. I tried webview_cef, which is based on the Chromium Embedded Framework. CEF spawns its own GPU process, and on Hyprland (a Wayland compositor based on wlroots) that conflicts with the compositor’s render pipeline. On my NVIDIA setup, the entire Hyprland session froze. Locked screen, no keyboard response, I had to kill it from a TTY. On top of that, CEF requires downloading a ~200MB binary on the first build. I gave up on CEF and wrote a native bridge in C++ with Claude using WebKitGTK and Flutter method channels. It worked, but every YouTube quirk required separate code for Linux and Android. just_audio also has no Linux desktop implementation. The Linux version turned into dead weight. I deleted ~1,500 lines of Linux-specific code and focused only on Android.

Then came the Samsung mic saga. On my Galaxy Z Fold, the mic was capturing an absurdly low signal. Peaks of ~0.005, basically silence as far as pitch detection was concerned. I spent two hours trying to figure it out. I lowered thresholds, raised software gain to 50x, disabled audio preprocessors. Nothing was working right. Until I figured out the real problem: Android’s AutomaticGainControl. The name says “automatic gain control,” which suggests it amplifies weak signals. In the Samsung DSP implementation, it does the opposite. It attenuates the signal to a low reference level, optimized for voice calls. With AGC on, the peak dropped from ~0.06 to ~0.003. Disabling AGC fixed it. But then the audio_session package was re-enabling AGC under the hood. I removed that one too. It was three rounds of fixes, each finding one more layer of the problem.

And the scoring. The scoring took longer than everything else combined. The first implementation used a cumulative average, which kept the score stuck at one value and never responded to live singing. I switched to a rolling window. Then the score was stuck at ~50% because of a bug in the primary score weight. I fixed it, and it started showing 70% even with nobody singing. Fixed it again. Streak mode wasn’t resetting properly during silence. The chromatic snap was giving high scores for anything. The pitch history wasn’t being cleared on silence gaps and the modes were going stagnant. Every fix revealed another bug. It took more than 25 commits just on the scoring, from the first prototype to the current state.

The result isn’t perfect. I know. But it works well enough to be fun, which was the goal from the start.

Settings

The settings panel lives behind the gear icon on the overlay. There are three mic presets for different environments (clean external mic, normal room, loud party), each adjusting confidence and amplitude thresholds. There’s a pitch shift for when the song is too high for your vocal range. The shift moves both the video audio and the scoring at the same time: it uses the HTML5 element’s playbackRate with preservesPitch=false, so +2 semitones speeds the audio up to 1.12x (pitch goes up) and -2 semitones slows it down to 0.89x (pitch goes down). The scoring compensates for the offset, so you sing in your comfortable range and the system grades you correctly. There’s mic calibration, a 3-second process that measures the room noise and adapts the thresholds. And there’s a restart to reset the score without reloading the video.

To switch scoring modes, tap the score box during playback.

Usage flow

1. Open the app. YouTube loads inside the app with the Frank Karaoke logo.
2. Search for a karaoke video. Any video works, but instrumental tracks with on-screen lyrics give better results.
3. The video pauses briefly to initialize the mic, download the song’s data for the pitch oracle, and prepare the overlay. The first time with a new song this takes 5-15 seconds. If you’ve played it before, it loads from cache instantly.
4. Sing. The “live” score reflects your current performance (exponential moving average with alpha 0.15, ~1 second response). The “overall” score is the cumulative average of the entire song.
5. When the video pauses, scoring pauses with it (so it doesn’t score ambient noise). If you seek, the score resets and gets a 5-second warmup.

How to install

The app isn’t on the Play Store yet, I’m waiting for Google to verify my developer identity. It should show up there in the next few days. In the meantime, it’s an open project and you can install it directly.

The easiest way is to download the signed APK directly from the GitHub releases page. On your Android phone or tablet, download FrankKaraoke-0.2.0-android.apk, open it and tap Install. If Android complains about “unknown sources,” enable it under Settings > Security for your browser. On the first run the app will ask for mic permission. Then go into settings (the gear icon) and calibrate the mic before singing — three seconds.

If you want to compile from source or contribute, the repository is on GitHub. You’ll need Flutter SDK 3.10+, Android SDK API 24+, and a physical device for mic testing (an emulator doesn’t give representative results).

git clone https://github.com/akitaonrails/frank_karaoke.git
cd frank_karaoke
flutter pub get
flutter run -d <device_id>

The README has the rest.

Stack: Flutter + Riverpod for state management, webview_flutter for YouTube, youtube_explode_dart for audio extraction, record for PCM mic capture, audio_decoder for reference decoding via Android MediaCodec, and the YIN algorithm implemented in pure Dart.

The technical documentation for the scoring system is in docs/scoring.md in the repository. It covers how SingStar, Joysound and DAM work, the academic papers, the pitch oracle architecture, the voice isolation problems on Android, and the roadmap.

The scoring is experimental

I have to be straight: the scoring system is experimental. Without per-song reference files, the evaluation is approximate. The app measures whether you’re in tune, whether you follow a melodic contour, whether your intervals are musical, whether you’re consistent. But it doesn’t tell you whether you’re singing the correct melody for this specific song (unless the pitch oracle manages to download the audio, and that doesn’t always work).

If you have experience with audio processing, pitch detection, or music evaluation, the repository is open and the research documentation in docs/scoring.md details what was tried, what works and what doesn’t. In particular: tuning the modes’ thresholds, improving voice isolation, and integrating with UltraSinger (which generates reference files from songs using Demucs + basic-pitch + WhisperX) are areas where contribution from people who know the subject would make a real difference. I’d appreciate any help from specialists on calibrating these systems.

Oh, and the name. Frank Karaoke. It’s a tribute to Sinatra. Who else?

Project on GitHub: github.com/akitaonrails/frank_karaoke

With that out of the way, let me explain why I built this.

The problem with the Brazilian press

I’m fed up.

Fed up of opening the newspaper and having to do mental gymnastics to separate information from narrative. Fed up of outlets like Folha de São Paulo, UOL, Carta Capital, Brasil 247, O Globo and several others that use misleading headlines, omit context on purpose, transpose evidence from other countries with no caveat, and create the appearance of consensus among outlets that are all saying the same thing because they’re following the same coordinated agenda.

This isn’t conspiracy theory. It’s a verifiable editorial pattern. And the worst part: most readers don’t have the time or the tools to notice. You read the headline, read the first two paragraphs, and walk away with the impression the article planted in your head.

What Frank Investigator does

You give it a news article URL. The system fetches the article with a headless Chromium (to get past paywalls and cookie walls), extracts the content while filtering out ads and sidebars, and decomposes the text into verifiable claims. Then the real analysis starts: divergence between headline and body, rhetorical fallacies (false causality, appeal to authority, strawman, bait-and-pivot), source distortion, temporal manipulation, selective citation, authority laundering. It expands the links cited in the article to verify whether the sources actually say what the author claims. It evaluates each claim with consensus from 3 AI models (Claude Sonnet 4.6, GPT-5.4, Gemini 3.1 Pro) through OpenRouter. It detects contextual gaps, coordinated campaigns across outlets, and measures the ratio between passion and evidence. 15 stages in total.

The central principle is “Truth Above Consensus”: a primary source (official data, government document, original academic study) vetoes any number of secondary sources repeating the same information. Ten newspapers repeating the same thing without a primary source still adds up to zero.

Let me show you five real examples.

Example 1: The Noelia Castillo case (BBC)

The BBC published a story with the headline “Noelia Castillo dies: a 25-year-old’s fight in the Spanish courts against her father to receive euthanasia.” At first glance it looks like a story about the right to euthanasia and a family dispute. A young quadriplegic woman who fought against her religious father’s opposition to exercise her right to die.

Except when you compare it with other outlets, like Veja, facts come up that the BBC almost completely omitted. And those facts change everything.

Noelia was taken from her family by the Spanish government at age 13 and placed under state custody. While she was under that custody, she suffered multiple gang rapes. The sexual violence resulted in serious psychiatric damage and a mental health history that already added up to 67% disability rating before the events of 2022. When she attempted suicide in October 2022 by jumping from the fifth floor of a building, she was left paraplegic. Her disability rating rose to 74%.

The euthanasia request was approved by Catalonia’s Guarantee and Evaluation Commission. The procedure was scheduled for August 2, 2024, but was suspended for over 600 days because of the father’s appeals. Five judicial instances ruled on it. The Constitutional Court dismissed any violation of fundamental rights. Spain’s Supreme Court denied the appeal. The European Court of Human Rights rejected the suspension request. On Friday, March 26, 2026, Noelia underwent euthanasia at the Sant Camil Residential Hospital, in Catalonia’s Garraf region.

But there’s a detail that Veja mentions that’s disturbing: Noelia reportedly expressed doubts before the procedure. And the hospital allegedly accelerated the process because her organs were already committed for donation.

The Frank Investigator report compared the coverage from several outlets.

The cross-analysis of the articles showed that some outlets, like the BBC, omitted facts that change the interpretation of the entire case. Others, like Veja, included the full context. The episode of gang sexual violence in October 2022, the criminal investigation, the psychiatric history since age 13, all of that appears in some coverage but is completely absent in others. And among the outlets that omitted these facts, none touched on the ethical question of whether the physical basis for the euthanasia request derives from a suicide attempt.

The convergent framing across outlets is one of “judicial battle,” “death she asked for,” “to stop suffering,” “to leave in peace,” softening the definitive nature of the procedure and positioning Noelia as the heroic protagonist and the father as the obstructive antagonist. The father, Gerônimo Castillo, and his Christian Lawyers are labeled “ultra-Catholics” or “ultra-conservatives” without any independent source backing that editorial classification.

Narrative coordination came in at 55%. It doesn’t seem to be active coordination between newsrooms, but thematic editorial alignment: everyone bought the autonomy-of-the-individual narrative without questioning it. What raises the score is the convergence of omissions. No outlet explains the legal distinction between the ECHR “actively authorizing” the euthanasia and simply refusing the provisional protective measures the father requested, which is what actually happened. The medical and bioethical implications of approving euthanasia in a case stemming from a suicide attempt show up nowhere. And any voice critical of the procedure is automatically framed as religious or ideological, never as medical or legal.

It’s the kind of case where the omission is the manipulation. The outlets that omitted these facts didn’t lie at any point. But by framing it as a “family dispute over euthanasia rights” and omitting the causal chain (state custody → gang rapes → psychiatric damage → suicide attempt → paraplegia → euthanasia), the reader walks away with an impression that’s radically different from reality. Comparing coverage is exactly the kind of thing the investigator does well: exposing what each outlet chose to show and what it chose to hide.

Example 2: “Government cuts taxes on nearly a thousand imports” (UOL)

UOL published that the government cut import taxes on nearly a thousand products, from medicine to hops. Positive headline, framed as a benefit to the consumer. Several other outlets ran the same thing with similar framing: the government did something good, prices are going down.

Except Gazeta do Povo tells the other half of the story. The government didn’t cut old taxes. What actually happened was: at some point before February 2025, the government raised import tariffs on more than 1,200 items, a measure that would have generated an estimated R$ 14 billion in revenue. Then, under social-media and public pressure, it partially backed down. Tariffs on around 970 capital goods, computing and telecommunications items were dropped to zero. Taxes on 120 IT products were reduced. And now they’re calling that a “tax cut.”

In other words: they raised taxes, took public pressure, partially walked it back, and rebranded it as a generous concession. The majority of the 1,200+ items that had tariffs raised still have higher tariffs than before. No final consumer prices went down. Prices went back to where they were for some products, and remain higher for most.

The Frank Investigator report cross-checked the articles and exposed what was omitted.

What the comparison between articles exposes: none of the outlets with the positive framing mentions that most of the 1,200+ items with raised tariffs still have higher tariffs after the two rounds of “cuts.” The fiscal impact on the R$ 14 billion revenue target from the original increases — nobody calculates it. Voices from the domestic industry that may be hurt by the tariff reduction on imported competitors — none. The Gecex’s objective criteria for defining “insufficient supply in the internal market” — they don’t show up.

The two articles analyzed build the same positive framing for the government. The paradox that the “cuts” are a partial reversal of increases made by the same government the previous year stays buried or simply absent.

It’s the classic kind of manipulation through reframing. Nobody lied. But “government cuts taxes” and “government backs down on tax hike after public pressure” describe the same event with opposite impressions. The editorial choice of which version to publish IS the manipulation.

Example 3: “Globo apologizes for PowerPoint” (Brasil 247)

This case blew up over the past few days. GloboNews showed a diagram on the Estúdio I program connecting President Lula, his ministers and Daniel Vorcaro, owner of Banco Master, who’s at the center of documented fraud. Globo later issued a public retraction, called the material “erroneous and incomplete,” and fired an editor.

Brasil 247 published a story with the headline “Globo apologizes for PowerPoint that tried to dump the Master case on Lula’s lap.”

The Frank Investigator report exposed what’s going on under the hood:

The central fact is real: Globo apologized and fired someone. But Brasil 247’s framing goes way beyond what the facts support. The headline saying “tried to dump it on Lula” attributes deliberate intent where the documents point to an editorial mistake. Globo’s retraction described the material as “erroneous and incomplete,” not as an attempt to incriminate anyone.

What stands out in this case is the coordinated campaign. The investigator gave it 62% narrative coordination.

Several outlets editorially aligned with the government used the phrase “without evidence” in a convergent way to describe the association between Lula and the Master case. All of them focused on Globo’s mistake as the central narrative point, instead of investigating the actual connections. No outlet mentioned which other political names were excluded from the original PowerPoint. None investigated Vorcaro’s documented connections with different spheres of power. The focus is meta-journalistic: they criticize the broadcaster instead of covering the scandal.

The fallacies detected: loaded language (“tried to dump,” “without evidence” used to frame an editorial mistake as a deliberate political attack), false causality (Globo’s retraction doesn’t prove the connections are false), cherry-picking (highlights the omission of names tied to the Lula government without contextualizing which other names were omitted), and bait-and-pivot (uses Globo’s apology as a hook to minimize the Banco Master scandal).

And the questions none of these outlets asked: which Supreme Court justices and politicians from other parties were also excluded from the PowerPoint? Does the Master case have documented connections to Lula government figures, or are they limited to previous administrations? Does Brasil 247, which is publishing this article, have a declared editorial alignment with the Lula government? What was the journalism Ethics Council’s reaction?

Overall confidence: 13%. The article doesn’t fabricate facts. But it selects, frames and omits in a way that builds a narrative the data doesn’t support.

Example 4: “Why expensive fuel is good” (Folha)

The economist Bernardo Guimarães published a column on Folha arguing that expensive fuel is good for society because it stimulates innovation in clean energy. He cites real academic articles (Popp 2002, NBER) and has verifiable academic credentials (PhD from Yale, professor at FGV EESP). It looks solid.

The full Frank Investigator report shows another picture.

The article is right to cite real studies. But it omits who pays the bill. The column completely ignores the distributive impact: low-income populations, residents of peripheral and rural areas, who depend more on personal vehicles and have less access to clean alternatives. For an economist, ignoring distributive effects is either incompetence or editorial choice.

There’s a worse problem. The empirical evidence he cites (US patents, electric vehicle data in California between 2014-2017) is transposed to Brazil with no caveat at all. Brazil has an ethanol matrix and flex-fuel infrastructure that completely changes the causal mechanism. The article treats it as if the Brazilian consumer were in the same situation as the Californian one, which is false.

And there’s the context the article mentions in passing but doesn’t develop: the war in Iran is making fuel prices rise around the world. Brazil should be in a privileged position thanks to the pre-salt and to ethanol. But decades of mismanagement and corruption at Petrobras mean we’re paying the same price as the rest of the world. Instead of questioning that, the column sells the idea that “at least it’ll stimulate clean energy.” It’s a rationalization of a problem that shouldn’t exist.

The investigator identified 5 questions the article refused to address, with 35% contextual completeness. The fallacies detected include false dilemma (presents expensive fuel as the only viable climate policy), cherry-picking (acknowledges short-term inelasticity but emphasizes only long-term effects), and loaded language (describes alternatives as “playing at planting a little sapling”).

Overall confidence: 25%. It isn’t fabricated disinformation. It’s opinion with cherry-picked evidence in favor of the thesis and omission of relevant counterpoints.

Example 5: “There’s no strong cinema without streaming regulation” (O Globo)

Renata Magalhães, president of the Brazilian Academy of Cinema, gave an interview in Miriam Leitão’s column on O Globo arguing that regulating streaming is a necessary condition for strengthening Brazilian cinema.

The Frank Investigator report found serious problems.

First: the central claim that “many films have low audiences” appears with no empirical data. No numbers, no historical series. It’s a pure authority claim with no analytical basis.

Second: there’s an internal contradiction the article doesn’t resolve. The text opens by saying that Brazilian cinema is “in international spotlight” (awards, festivals). And then immediately argues that the absence of regulation prevents the industry from being strengthened. But if Brazilian cinema is already winning international awards without that regulation, the argument that the regulation is a necessary condition collapses. The article doesn’t address that contradiction.

And here’s the elephant in the room that none of these articles mentions: people simply don’t want to watch most of these films. Internationally awarded Brazilian cinema is made to compete in Cannes and at the Oscars, not to fill movie theaters in Brazil. Instead of asking why the Brazilian audience isn’t interested, the industry would rather ask for streaming regulation to force platforms to fund and screen content that has no spontaneous audience. It’s the classic playbook: use public money and regulation to keep alive an industry that doesn’t sustain itself in the market.

The interviewee is the president of the Brazilian Academy of Cinema. She has a direct institutional interest in the regulation. The article doesn’t present any opposing voice and doesn’t discuss the costs to consumers: subscription price increases, catalog reduction. A single source, with a declared conflict of interest, with no counterpoint.

And here comes the coordinated campaign, with 55% narrative coordination. Multiple outlets reproduce the same emotional framing: cinema “at risk,” regulation as “essential.” None of the identified sources discusses comparable empirical international evidence on the effectiveness of content quotas (Europe has experiences with contradictory results). None mentions the Brazilian Academy of Cinema’s conflicts of interest. The fact that the internationally awarded Brazilian productions were made without the proposed regulation is omitted convergently across all outlets. Only one isolated site (targethd.net) mentioned negative impacts on consumers.

Overall confidence: 9%. The article is legitimate editorial advocacy, but with analytical flaws that limit its informational value to nearly zero.

What the investigator analyzes

The 5 examples above show different patterns, but the analysis criteria are the same.

The strongest signal of manipulation is omission. It isn’t what the article says that misleads, it’s what it leaves out. The contextual gap analysis identifies the questions the article should have answered and didn’t, and searches for counter-evidence on each one. In the examples above, articles that omit most of the relevant context aren’t informing anyone. And when cross-comparison between outlets shows that some covered the facts and others didn’t, like in the Noelia case or the tax cut, it’s hard to argue that the omission was accidental.

Then comes the detection of coordinated campaigns. Several newspapers covering the same subject is normal. All of them using the same loaded language, focusing on the same points and omitting the same counterpoints at the same time isn’t. The strongest signal of coordination isn’t what outlets say in common, but what they omit in common.

There’s also reframing, which is more subtle. In the tax case, the government raised tariffs, backed down under pressure, and the outlets called it a “cut.” Nobody lied technically, but the choice of framing completely changes the interpretation. This kind of manipulation is harder to detect because each individual statement is defensible.

The rhetorical fallacies catch specific constructions: false dilemma (“either you regulate or cinema dies”), bait-and-pivot (open with a positive fact and pivot to a crisis narrative), loaded language (“without evidence” used to attribute intent). Each detected fallacy comes with the exact citation of the passage and the explanation of why that construction is problematic.

And there’s the principle that ties it all together: if 10 outlets repeat the same claim citing each other, the LLM consensus has to reflect that the chain of evidence is circular, not that the claim is well supported. Volume of coverage is not a proxy for truth.

Why you can’t “just ask ChatGPT”

The first reaction many people will have is: “why do I need this if I can just paste the article into ChatGPT and ask it to analyze?”

Try it. Take a political article and ask ChatGPT to criticize it. It’ll criticize. Take the same article and ask it to confirm. It’ll find arguments to confirm. The LLM isn’t searching for truth. It’s predicting which response you most likely want to hear given the framing of your question. If you ask “analyze the problems with this article,” the model will find problems. If you ask “is this article correct?”, it’ll find merits. It’s automated confirmation bias.

There’s another problem. General-purpose LLMs are trained to be agreeable. The sycophantic tendency (agreeing with the user) is documented in every large model. If your conversation history indicates you’re left-leaning, the model tends to frame answers in a way that pleases that profile. If you’re right-leaning, same thing in the other direction. It’s not lying on purpose. It’s optimizing for user satisfaction, which is literally the metric it was trained for via RLHF.

And worse: LLMs hallucinate. If they don’t have enough evidence to support the answer they think you want to hear, they invent it. They fabricate plausible citations and data, attribute statements to people who never made them. If you ask it to criticize an article about fuel, it might invent a fictional study that “proves” the opposite of the article. It sounds convincing. But it doesn’t exist.

Frank Investigator was built precisely to avoid these problems. The first design decision is that no human asks the LLM a question. There’s no open-ended prompt like “analyze this article.” Each step in the pipeline has structured prompts that ask for specific analyses: “list the rhetorical fallacies in this passage,” “identify which contextual information is missing,” “compare the headline with the body.” The model doesn’t know whether the operator agrees or disagrees with the article, because the operator never expresses an opinion. That eliminates confirmation bias at the root.

To deal with hallucination, every analyzer that uses an LLM includes the instruction “CRITICAL — NO HALLUCINATION: Only reference URLs, sources, claims, quotes, and data that are EXPLICITLY present in the input provided to you. Do not invent, guess, or fabricate any URL, source name, statistic, quote, or claim. If you cannot verify something from the provided text, mark it as unverifiable — never fill in details.” It doesn’t eliminate hallucination completely, but it cuts it down a lot. And since 3 models from different companies (Anthropic, OpenAI, Google) answer the same questions, when one hallucinates the other two usually disagree. The consensus is weighted by confidence, not by simple majority. If two models say “supported” with 70% confidence and one says “mixed” with 95%, the “mixed” weighs more. The further apart the disagreement, the bigger the penalty on final confidence. If one model starts giving inconsistent answers, it gets put in quarantine and the other two carry on.

But the safeguard I consider most important is the primary source veto. If a primary source (IBGE data, court ruling, original study) contradicts a claim, confidence is capped at 60% and the verdict is forced to “mixed,” even if all 3 LLMs say “supported.” Ten newspaper articles repeating a claim don’t override one official datum that contradicts it. Along the same lines, if 5 stories “confirm” a claim but they all come from the same editorial group (Folha/UOL, Globo/G1/Valor), the system knows they’re the same voice and reduces the weight. Volume doesn’t replace independence.

None of this makes the system perfect. But it’s categorically different from pasting text into ChatGPT and asking “what do you think?”.

The numbers

Stack: Ruby 4.0.1, Rails 8.1.2, SQLite with WAL mode, Solid Queue (jobs inside Puma), Solid Cable (WebSockets), Tailwind CSS v4, headless Chromium via Ferrum CDP, deploy with Kamal to GitHub Container Registry. AGPL-3.0.

The development process

The project has 129 commits in 9 active days of work (March 11-16 and 25-27). The first day was the heaviest: more than 60 commits on March 11 alone, going from zero to a working system with content extraction, claim decomposition, LLM evaluation, and a web interface with live updates via Turbo Streams.

The commits tell the story. It started with the foundation:

1e32d5a - Initial Frank Investigator foundation
564eb97 - Add recursive source crawling and RubyLLM scaffold
c8c5357 - Add Brazil source registry and authority connectors
3d30617 - Add U.S. authority profiles, source role modeling, and specialized connectors

Then came the misinformation analyzers, one by one:

a65fef7 - Add rhetorical fallacy analyzer for detecting narrative manipulation
5dcf99d - Add headline-body divergence detection and headline citation amplification
a113efd - Add smear campaign defense with circular citation and viral volume detection
56d501a - Add media ownership modeling, syndication detection, and independence analysis

The hardening phase against noise and false positives is the one that took the most time:

46e4bc5 - Add pre-fetch defenses: URL filtering, circuit breaker, fetch prioritization
168fea1 - Add post-fetch content gate, claim noise filter, and duplicate content skip
b2843d7 - Add paywall detection, pricing noise filter, and ofertas.* host rejection
b1b230d - Rewrite Chromium fetcher with Ferrum CDP for anti-bot evasion

Then came the interface, deployment, and the more advanced analyzers:

c5afcea - Replace custom CSS with Tailwind CSS v4 and rewrite all templates
95a8ec4 - Add Docker deployment with bin/deploy script
ccfacc9 - Add coordinated narrative detection across media outlets
ba3e2f4 - Add 6 new misinformation detection analyzers with cross-analysis
fda984b - Add LLM-generated investigation summary with quality assessment
a4ebe78 - Add contextual gap analysis to detect manipulation through omission

And in the most recent commits, simultaneous 3-model consensus and test optimization:

5cc9c05 - Enable 3-model LLM consensus: Sonnet 4.6, GPT-4.1 Mini, Gemini 2.5 Pro
cdf5fb5 - Batch 5 content analyzers into single LLM call, add anti-hallucination
28c305e - Add WebMock stubs for LLM and web search — tests run in 1.4s (was 540s)

The tests that used to run in 9 minutes now run in 1.4 seconds with WebMock stubbing of every LLM and web search call. That made a huge difference in iteration speed.

Limitations

News fact-checking is a hard problem. A single article doesn’t contain everything needed for a complete analysis. The author chose what to include and what to omit, and that choice is already the first level of manipulation. The sources cited inside the article were selected by the author to support their narrative, not to give a balanced view.

Frank Investigator doesn’t tell you what’s true and what’s false. The result is a report with strong and weak points, not a “true” or “false” stamp. Even with all the safeguards I described above, the counter-evidence searched for automatically may not be the most relevant. The fallacies detected can have false positives. The coordinated campaign analysis depends on what web search returns at the moment of the lookup.

Use the reports as a starting point to form your own opinion, not as a final verdict.

The project is open source, AGPL-3.0. If you want to contribute, test, report bugs or suggest improvements: GitHub. If you want to follow the analyses, the The Makita Chronicles newsletter is going to have a “Notícias Duvidosas” (Dubious News) section with the summary and a link to the full report of each investigation.

It “works,” but this project was more about the exercise. With that out of the way, let me tell you why I did it.

The problem with OpenClaw

OpenClaw is a gateway that connects messaging channels (Telegram, Discord, Slack, WhatsApp, etc.) to AI providers (OpenAI, Anthropic, Ollama). You configure it, bring up the server, and you can chat with LLMs straight from Telegram or any other channel. It’s a popular project with plenty of activity.

Too much activity, actually.

I did a depth-1 clone of the repo and ran tokei:

Over a million lines of TypeScript. The 2,799 test files sound like a lot in absolute numbers, but proportional to the codebase size, the coverage is low. Most of the code lives in 29 packages of a monorepo with 21 channel extensions.

I went looking for more commits to understand the development pace. In the 100 commits I managed to pull, all of them landed in just 2 days (March 9 and 10). ~50 commits per day, from 42 different contributors. Vibe coding to the extreme.

The conclusion is the one you’re imagining: enormous volumes of AI-generated code being dumped into a repository at a speed that makes serious human review impossible. And that bothered me enough to go investigate further.

The security audit

I asked Claude to do a complete security audit of OpenClaw’s code. The report found:

Seven critical vulnerabilities. Let me list a few:

• Timing side-channel in token comparison — safeEqualSecret() does an early return on the type-check, letting an attacker distinguish malformed tokens from wrong tokens by measuring latency.
• eval() in the browser tool — arbitrary JavaScript execution with no sandbox.
• Shell with no allowlist — any tool can run any command on the system.
• Slack webhooks with no signature verification at all.
• Transcripts and config in plaintext on disk, no encryption.
• No effective rate limiting — IPs can be spoofed if the operator configures trusted proxies broadly.

Those are just the ones Claude found in an automated scan. There are probably more.

I’m not going to run that on my machine. Not even inside a Docker container. A gateway that receives webhooks from the internet, executes shell commands, connects to AI APIs with your keys, and stores conversation history, all of it with 7 known critical vulnerabilities? No, thanks.

So I did what any rational developer would do: I decided to build my own.

The first attempt: Claude Code

I started the most direct way. Cloned OpenClaw, pointed Claude Code at the code, and asked: “rewrite this in Rust.”

Didn’t work. The codebase is too big. Over a million lines of TypeScript spread across 29 packages. Claude can’t keep all of it in context. The initial result was very incomplete: lots of types created but no implementation, todo!() everywhere, too much boilerplate and not enough functionality.

I switched to Codex 5.4 to test. Same thing: I asked it to analyze and rewrite. It improved a bit in certain aspects, but the fundamental problem is the same. No AI today takes a project of that size and rewrites it in one go. The context doesn’t fit.

The technique that actually works

What works is going slowly. One step at a time.

Ask Claude (or Codex) to analyze the original code in stages. Make a long plan detailing each feature. Then implement one feature at a time in Rust, with tests, commit, and repeat. It’s tedious, but it produces code that compiles and actually works.

The reason is simple: the original code is so massive that no AI agent (and not even several in parallel) can keep all of it in context at the same time. You have to decide what matters, implement that, validate, and move on to the next thing.

And you have to decide what to cut. OpenClaw has 21 channel extensions: Google Chat, iMessage, IRC, Teams, Matrix, Mattermost, Nostr, Twitch… I don’t need any of those. I kept the mainstream channels: Web, Telegram, Discord, Slack, Signal, WhatsApp and Email. TTS? Out. Polls? Out. WhatsApp Web through Baileys? Out, I use the official Cloud API. They’re features that add complexity without proportional value.

Discovering IronClaw

In the middle of development, I came across IronClaw, which presents itself as “OpenClaw in Rust.” Great, I thought. Let me see what they did.

I cloned the repo and asked Claude to write a comparison report. IronClaw has good things. I adopted 12 features:

Circuit breaker with retry and exponential backoff for LLM provider resilience. Credential leak detection in the output. LLM response cache with SHA-256 of the prompt. Cost tracking with budget guards (warning at 80%, block at 100%). Extended thinking for Claude 3.7+ and o1. MCP client for external tool servers. Lifecycle hooks on inbound, tool calls and outbound. Smart model routing that sends simple queries to cheaper models. Tunnel support (cloudflared, ngrok, tailscale). Interactive REPL (frankclaw chat). Routines with event triggers beyond cron. Job state machine with auto-repair.

But IronClaw depends on PostgreSQL + pgvector, has a WASM sandbox (wasmtime adds ~10MB), and is part of the NEAR AI ecosystem. I want a single binary with embedded SQLite and zero external dependencies.

What FrankClaw brings from OpenClaw

The OpenClaw core is there: 7 messaging channels, multi-provider with failover, agent runtime, command system, skills, subagents, browser automation through CDP, bash tool with allowlist, cron jobs, interactive REPL. Plus the 12 IronClaw features I listed above.

But several pieces had to be rewritten in a way the original didn’t. Context compaction, for example, uses a sliding window with token estimation, message pruning and automatic repair of tool pairs that get orphaned when the context is cut. Provider failover is now model-aware: if you ask for a Claude model and the current provider is OpenAI, it skips automatically instead of erroring out. The canvas renders SVG, HTML and Markdown with revision conflict detection. Things that in OpenClaw either didn’t exist or were half-built.

Then came the phase of adding what was missing for parity. FrankClaw today has 30+ LLM tools: web_fetch (SSRF-safe with HTML-to-text), web_search (Brave API), file_read/file_write/file_edit (sandboxed in the workspace with path traversal protection), pdf_read, image_describe (through vision models), audio_transcribe, sessions_list/sessions_history/sessions_delete, message_send/message_react, cron_list/cron_add/cron_remove, config_get (auto-redacts secrets), agents_list, memory_get/memory_search, plus the browser tools (browser_open, browser_extract, browser_snapshot, browser_click, browser_type, browser_wait, browser_press, browser_sessions, browser_close, browser_aria).

Some additions that don’t exist in the original OpenClaw: a memory/RAG system with SQLite FTS5 and embeddings (OpenAI, Gemini, Voyage) that automatically syncs workspace files. An OpenAI-compatible API (/v1/chat/completions and /v1/models), so any client that speaks that protocol (Cursor, Continue, Open WebUI) can use FrankClaw as a backend with no adaptation. A ratatui TUI for people who prefer the terminal. Interactive approval of destructive tools before execution.

Smaller things that make a difference in practice. You can configure multiple API keys per provider with round-robin and automatic backoff, so if one key hits the rate limit, the next one takes over. The model catalog already knows context windows and costs of OpenAI and Anthropic models without you having to configure them. URL extraction from messages has a private IP blocklist against SSRF. The command system accepts inline directives (/think, /model) in addition to aliases.

On the operational side: ACP (Agent Client Protocol) over JSON-RPC 2.0 on top of NDJSON for people who want to integrate programmatically. Plugin system with manifests and enable/disable lifecycle. i18n with 9 locales via FRANKCLAW_LANG. Workspace identity files (SOUL.md, IDENTITY.md) to define the bot’s personality per project. Channel health monitor with auto-restart. WebSocket with ping keepalive that survives proxy and tunnel timeouts. frankclaw start/stop/status for people who want to run it as a daemon with PID tracking. And the entire configuration migrated from JSON to TOML.

Hardening: where the real difference is

The audit report we ran on OpenClaw found 7 critical and 9 high vulnerabilities. FrankClaw fixes all of them:

FrankClaw compiles with #![forbid(unsafe_code)] in all 13 crates. Zero unsafe blocks.

And the audit didn’t stop at OpenClaw. We did a per-component audit in 14 phases comparing each part of FrankClaw against the original: channels, providers, runtime, tools, sessions, crypto, cron, webhooks. All documented.

Deploy: how to install

FrankClaw runs with Docker Compose. Three containers: gateway, headless Chromium (for browser tools), and Cloudflare tunnel (to receive webhooks).

1. Clone and configure

git clone https://github.com/akitaonrails/frankclaw.git
cd frankclaw
cp .env.docker.example .env.docker

Edit .env.docker with your keys:

# Providers de IA (preencha os que usar)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=

# Canais (preencha os que quiser usar)
TELEGRAM_BOT_TOKEN=         # via @BotFather
WHATSAPP_TOKEN=             # Meta Business Platform
WHATSAPP_PHONE_ID=
WHATSAPP_VERIFY_TOKEN=
DISCORD_BOT_TOKEN=
SLACK_BOT_TOKEN=
SLACK_APP_TOKEN=

# Embedding providers (só se usar memória/RAG)
# GEMINI_API_KEY=
# VOYAGE_API_KEY=

# Opcional: criptografia de sessions (recomendado)
# Gere com: openssl rand -base64 32
FRANKCLAW_MASTER_KEY=

# Opcional: scan de malware em uploads
VIRUSTOTAL_API_KEY=

2. Configure the gateway

The frankclaw.toml file defines agents, models and channels. Use the wizard or the examples:

# Generate a base config with the web channel
cargo run -- onboard --channel web

# Or copy from the examples
ls examples/channels/

For each channel, the CLI has ready-made templates:

cargo run -- config-example --channel telegram
cargo run -- config-example --channel whatsapp

3. Cloudflare Tunnel (to receive webhooks)

If you’re going to use channels that need a webhook (Telegram, Discord, Slack, WhatsApp), you need a public tunnel. The Docker Compose already includes cloudflared:

# Copy your Cloudflare credentials
cp docker/cloudflared/config.yml.example docker/cloudflared/config.yml
cp ~/.cloudflared/<tunnel-id>.json docker/cloudflared/credentials.json
cp ~/.cloudflared/cert.pem docker/cloudflared/cert.pem
# Edit config.yml with your hostname

4. Bring it up

docker compose up -d

The gateway comes up on port 18789 (internal to Docker). cloudflared routes external traffic. Chromium stays on the internal network for browser tools.

To test locally without Docker:

cargo run -- chat

Opens the interactive REPL straight in the terminal (it now also has a ratatui TUI with dark mode and tabs). No gateway, no webhook. Good for validating that the AI provider is responding before configuring channels.

Validation

cargo run -- check     # validates config
cargo run -- doctor    # full diagnostic
cargo run -- audit     # security audit with severity ratings

audit is the one I like the most. It checks whether you have encryption enabled, whether file permissions are correct, whether webhooks have signature verification, whether the bash tool is in deny-all. It exits with a non-zero exit code when it finds critical issues, so you can drop it in CI.

The development process

The project has 178 commits in ~5 days of work (March 10-16). Almost 57 thousand lines of Rust in 120 files, organized in 13 crates.

The commits tell the story. The first few dozens were scaffolding: workspace structure, basic types, the HTTP/WebSocket gateway, first version of the channel adapters. Mass-generated code, lots of incomplete pieces.

Then the channel adapters started, one by one:

236aa1c - Minimal Discord channel adapter
fd67017 - Minimal Slack channel adapter
9f51373 - Minimal Signal channel adapter
1052e47 - WhatsApp channel webhook adapter
035f86e - Email channel adapter (IMAP inbound, SMTP outbound)

When the basic channels were in place, the IronClaw integration came in one big commit:

c87ab32 - IronClaw-derived features: circuit breaker, retry, leak detection, cache, cost tracking, extended thinking

And then came the hardening. That was the phase where I had to step in manually the most, because Claude generates functional code but doesn’t think about attack vectors on its own:

db34198 - Prompt injection sanitization, external content wrapping, prompt size limit
5719b34 - Optional VirusTotal malware scanning for file uploads
ccd2b2b - Harden input validation across all user-facing entry points
aa918ee - Optional ai-jail sandbox for bash tool
2d7b1df - Security audit command with severity-rated findings
d12cc97 - 3-tier ToolRiskLevel system replacing binary browser mutation flag
21e0c91 - Timing-safe token comparison in WhatsApp, crypto audit tests
e240c1b - Webhook replay prevention with timestamp verification
876a78c - Gateway & media: SSRF redirect validation, filename hardening

22 security hardening commits. Plus 10 more component audits. Each finding became a commit with a fix.

Then came the per-channel audits, each one uncovering different edge cases:

43b085f - Discord audit: HELLO timeout, fatal close codes, message chunking
12c7cff - Telegram audit: caption overflow, parse fallback, edit idempotency
f515062 - WhatsApp audit: message type filtering, send error classification
3c42aff - Slack audit: fatal auth errors, send error classification

Browser tools needed extra attention. A headless Chrome that gets URLs from an LLM is an obvious attack vector:

3217a96 - Browser automation: CDP timeout, SSRF guard, session limits, crash recovery
d98a803 - Gate mutating browser tools behind operator approval
014f56e - Browser screenshot/ARIA tools for accessibility tree inspection

In the more recent commits, the project started diverging from OpenClaw:

5d73c4d - OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models)
d832c36 - Memory/RAG system with SQLite FTS5, embeddings, and file sync
2b05f47 - Interactive tool approval for mutating/destructive tools
49034eb - Web console: dark mode, 8 tabs, focus mode, tool sidebar
9f51a18 - TUI, Gemini/Voyage embeddings, plugin management, ACP protocol
3c0703b - Config migration from JSON to TOML
eb130ad - Channel health monitor with auto-restart and rate limiting
2c3ea7e - Workspace bootstrap files (SOUL.md, IDENTITY.md) to system prompt
9df61bf - Model-aware failover routing, canvas SVG rendering
c7ba108 - WebSocket ping keepalive and auto-reconnect to web console

The OpenAI-compatible API is the one I use the most day to day. Cursor, Continue, Open WebUI, anything that speaks the OpenAI protocol can use FrankClaw as a backend without touching anything on the client side.

The numbers

Compared to OpenClaw:

The numbers aren’t directly comparable. OpenClaw has 21 channel extensions I cut, a more complete web UI, and niche features I didn’t port. But the core (gateway, mainstream channels, providers, runtime, sessions, tools, memory) is there with 22x less code.

How to help (beta testing)

FrankClaw works for basic conversation through Web and Telegram (I tested both). WhatsApp works for simple messages. Discord, Slack, Signal and Email are implemented but haven’t had extensive testing. We haven’t done any complex workflows yet.

If you want to test it: clone the repository, bring it up with Docker Compose, configure at least one channel (Telegram is the easiest) and try to use it normally. Send messages, test tool calls, try to break the system. Open Issues on GitHub with whatever you find.

What I know needs more eyes: workflows with tools (browser, bash, MCP), subagent orchestration, failover between providers, session persistence with encryption, smart routing between models, scheduled jobs, the memory/RAG system, and the OpenAI-compatible API. Basically everything that goes beyond “send a message, get a reply.”

You don’t have to be a Rust developer. The bigger value is in using the system in ways I didn’t think of and finding the edge cases that only show up under real use.

FrankClaw doesn’t replace OpenClaw today. OpenClaw has more channels, more features, more people working on it. But it carries the weight of over a million lines of TypeScript generated at 50 commits per day by dozens of contributors, with documented critical vulnerabilities. FrankClaw is the alternative for anyone who looks at that and thinks “I’m not running this code on my machine.”

But I’ll be honest: as fun as it was to build, I personally don’t know if I need this. FrankClaw is a generic gateway, designed to be flexible, to connect any channel to any provider, with an agent runtime, tools, subagents, jobs, hooks. It’s a lot of infrastructure.

What I’ve discovered over the last few months is that I can build custom bots for specific tasks much faster. In 1 day I have a bot working, focused on what I need, without carrying the weight of a generic framework. That’s what I did with Marvin on the newsletter project, for instance. A custom-built Discord bot that does exactly what I need and nothing else.

A generic gateway like FrankClaw makes more sense for someone who wants a unified interface between several chat channels and several AI models without coding. If that’s your case, give it a shot. If you’re a developer and you know what you want, maybe a custom bot will serve you better. Up to you.

The repository is here. AGPL-3.0.

But the third project is a different beast. It’s a security project. And the motivation comes from an old pain.

The problem: too many emails

As an ex-YouTuber and content creator, I get an absurd amount of email. Event invitations, collab proposals, sponsorship offers, requests to promote stuff. Every kind of pitch.

I delete 100% of them. I don’t read them. Most of them I mark as SPAM without even opening. The easiest way to get reported by me is to send me an email — I don’t care, because I don’t need to. I also don’t answer the phone. Ever. And I automatically block anyone who messages me directly on WhatsApp, regardless of the content. My time is too valuable to spend triaging messages from strangers. I delete everything and move on.

It works for me. But I know most people don’t operate that way.

The poison is VANITY

Most people who fall for email phishing don’t fall because of technical ignorance. They fall because of VANITY.

“Hello, we’d like to invite you to our exclusive event.” “Your brand has been selected for a special partnership.” “Congratulations, you’ve been nominated as a reference in your sector.”

The dopamine hits before reason has a chance to step in. Someone recognized your work, someone wants to give you money. That’s exactly when the scammer gets you.

It isn’t the multinational CEO who falls for the Nigerian prince scam. It’s the micro-influencer who gets a sponsorship offer that’s “too good to be true.” It’s the small business owner who gets an invite to an event that looks legitimate. Vanity turns off critical thinking.

And the scams are getting more sophisticated by the day. With LLMs, any criminal can generate perfect emails in Portuguese, with no grammar errors, professional formatting, and domains that mimic real companies. The old “look at the typos” doesn’t work as a filter anymore.

The idea: Frank FBI

Instead of trying to teach everyone how to spot phishing (which doesn’t work, because vanity beats training), why not build a tool that does it automatically?

Got a suspicious email? Forward it to a dedicated address. In a few minutes, you get back a detailed report with everything the tool managed to find out about that email.

That’s how Frank FBI was born — Fraud Bureau of Investigation.

How it works

You set up a dedicated Gmail account (any Gmail with an App Password works). You register the email addresses authorized to use the service. From there, the flow is:

1. You receive a suspicious email in your personal inbox
2. You forward it to the Frank FBI address
3. The system analyzes it automatically
4. You receive the response in the same thread, with the full report

To give you a sense of the result, here’s a report for a legitimate email (cold outreach from a real company):

And here’s a suspicious one, where the sender’s domain tries to impersonate another company:

Frank FBI runs 6 layers of analysis, each with a specific weight in the final score:

Layer 1 — Header authentication (15% weight)

SPF, DKIM, DMARC. Does the Reply-To match the From? Are there suspicious anti-spam headers? This layer answers the most basic question: did the email actually come from who it claims to?

Layer 2 — Sender reputation (15% weight)

Looks up the domain’s age via WHOIS (a domain registered last week is already a signal), checks whether the IP is on DNS blacklists (DNSBL), and maintains a local reputation database that improves as more emails are analyzed. If the sender pretends to be a company but emails from a Gmail, that weighs in.

Layer 3 — Content analysis (15% weight)

This is where pattern matching comes in: artificial urgency (“your account will be locked in 24 hours”), requests for personal data, authority impersonation, financial offers. It also detects URL shorteners and links where the displayed text doesn’t match the real href — that classic “click here” pointing at a completely different domain. Checks for dangerous attachments (.exe, .scr, Office macros).

Layer 4 — External APIs (15% weight)

URLhaus (abuse.ch) maintains a known-malicious URL database. VirusTotal aggregates results from dozens of antivirus engines. If any URL in the email has already been flagged as malware or phishing by these databases, the score goes up. Results are cached with a TTL so we don’t blow through the rate limits of the free APIs.

Layer 5 — Entity verification (10% weight)

This is the layer I find the most interesting. Frank FBI does OSINT — Open Source Intelligence. It uses Brave Search to verify whether the sender or company actually exists. Does WHOIS on the domain directly. Captures a screenshot of the site with headless Chrome. Cross-references all of it to try to answer: “is this entity real and is it who it claims to be?”

Here’s a real example of this layer in action, analyzing an email that tried to impersonate a legitimate company:

The domain was 83 days old, registered through Namecheap, with no verifiable online presence. The system found discrepancies between the name in the email and the public records, and identified that the legitimate company’s real domain was a different one.

Layer 6 — LLM analysis (30% weight)

Queries 3 AI models in parallel: Claude Sonnet (Anthropic), GPT-4o (OpenAI) and Grok 4 (xAI), via OpenRouter. Each model analyzes the email independently. A confidence-weighted consensus system combines the results. If all 3 agree it’s fraud, confidence is high. If they disagree, the system weighs them. If every LLM fails, it falls back to a neutral score of 50 instead of guessing — better to admit ignorance than to hallucinate.

The final score is a confidence-adjusted weighted average. Possible verdicts: Legitimate (0-25), Suspicious OK (26-50), Suspicious Fraud (51-75) or Fraudulent (76-100). There’s also a risk escalation policy that imposes minimum floors: if critical indicators were detected (confirmed malicious URLs, DKIM failure), the score can’t fall below certain thresholds, even if the other layers found nothing.

Self-hosted: your data stays with you

Frank FBI is self-hosted. You run it on your own server. Your emails don’t pass through any third-party service (except the verification APIs like VirusTotal, which only receive URLs, not the email’s content).

You can install it on your home server, like I did, or inside your company’s infrastructure for your employees to use. The deploy is via Docker Compose with 4 containers: the Rails app, a background job worker, an IMAP poller, and an initial database setup. Bring everything up with a docker compose up -d and it’s running.

Reporting fraud to the community

Beyond analyzing emails for you, Frank FBI can optionally report confirmed fraud to community databases. This only happens when the score is >= 85 and the verdict is “fraudulent.” It’s opt-in.

ThreatFox (abuse.ch) is an open database of indicators of compromise (IOCs) maintained by the security community. When you report a malicious URL or domain there, firewalls, email filters and SIEMs around the world can consume that information to block the threat.

AbuseIPDB is the same idea, but for IPs. If the IP of the sender of the fraudulent email gets reported there, email providers and network admins can block malicious traffic before it reaches users.

And SpamCop is one of the oldest spam reporting services. Frank FBI forwards the full email to SpamCop, which analyzes the headers and reports to the responsible providers. It’s reporting directly to whoever can act.

Each report is a contribution so other people don’t fall for the same scam. And it’s automated: if the email is clearly fraudulent, the report goes out without manual intervention.

But reporting wrong things does damage. That’s why the system has anti-poisoning protections: a list of ~40 known domains (Microsoft, Apple, Google, Amazon, PayPal, governments) that are never reported; domains with clean scans are excluded; cloud infrastructure IPs (Google, Microsoft, Cloudflare) are filtered; free email domains are ignored. Only genuinely malicious IOCs reach the community databases.

Deployment: how to get it running

Let me explain how I deployed it on my home server. If you have a VPS, NAS or any Linux machine with Docker, the process is the same.

1. Clone and build the image

You need a private registry to store the Docker image. I use Gitea on my home server — it’s a lightweight self-hosted GitHub that includes a container registry. If you already have a private GitHub, you can use the GitHub Container Registry (ghcr.io) instead.

# Clone the repository
git clone https://github.com/akitaonrails/frank_fbi.git
cd frank_fbi

# Build the Docker image
# If using Gitea (replace with your registry's IP/port):
docker build -t seu-servidor:3007/frank_fbi:latest .
docker push seu-servidor:3007/frank_fbi:latest

# If using GitHub Container Registry:
docker build -t ghcr.io/seu-usuario/frank_fbi:latest .
docker push ghcr.io/seu-usuario/frank_fbi:latest

2. Configure the .env

Copy .env.example and fill it in. The required variables:

# Gere com: ruby -rsecurerandom -e 'puts SecureRandom.hex(64)'
SECRET_KEY_BASE=

# Gere com: bin/rails db:encryption:init (roda local, copia os 3 valores)
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=

# Gmail dedicado pro Frank FBI (crie uma conta só pra isso)
# Ative 2FA e gere um App Password em https://myaccount.google.com/apppasswords
GMAIL_USERNAME=seu-frank-fbi@gmail.com
GMAIL_PASSWORD=xxxx-xxxx-xxxx-xxxx
GMAIL_IMAP_HOST=imap.gmail.com
GMAIL_SMTP_HOST=smtp.gmail.com

# Senha do ingress do Action Mailbox (qualquer string aleatória)
RAILS_INBOUND_EMAIL_PASSWORD=uma-senha-qualquer-longa

# LLM via OpenRouter (obrigatório pra Camada 6)
OPENROUTER_API_KEY=sua-chave-openrouter

# Seu email de admin (pra gerenciar remetentes autorizados)
ADMIN_EMAIL=seu-email@pessoal.com

The external APIs are optional but recommended. Without them, the corresponding layers simply don’t run:

# VirusTotal (grátis, 4 requests/min) - https://virustotal.com
VIRUSTOTAL_API_KEY=

# WhoisXML (grátis, 500 requests/mês) - https://whoisxmlapi.com
WHOISXML_API_KEY=

# Brave Search (grátis, 1 req/s) - https://brave.com/search/api/
BRAVE_SEARCH_API_KEY=

And the community reporting, which is fully opt-in. Leave it blank if you don’t want to report:

THREATFOX_AUTH_KEY=
ABUSEIPDB_API_KEY=
SPAMCOP_SUBMISSION_ADDRESS=

3. Docker Compose on the server

Create the docker-compose.yml on your server. Replace the image with your registry:

services:
  setup:
    image: seu-servidor:3007/frank_fbi:latest  # ou ghcr.io/seu-usuario/frank_fbi:latest
    command: ["./bin/rails", "db:prepare"]
    env_file: .env
    environment:
      - RAILS_ENV=production
    volumes:
      - ./storage:/rails/storage

  app:
    image: seu-servidor:3007/frank_fbi:latest
    env_file: .env
    environment:
      - RAILS_ENV=production
    volumes:
      - ./storage:/rails/storage
      - ./emails:/rails/emails
    depends_on:
      setup:
        condition: service_completed_successfully
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/up"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 15s
    restart: unless-stopped

  worker:
    image: seu-servidor:3007/frank_fbi:latest
    command: ["./bin/jobs"]
    env_file: .env
    environment:
      - RAILS_ENV=production
    volumes:
      - ./storage:/rails/storage
    depends_on:
      setup:
        condition: service_completed_successfully
    restart: unless-stopped

  mail_fetcher:
    image: seu-servidor:3007/frank_fbi:latest
    command: ["./bin/rails", "frank_fbi:fetch_mail"]
    env_file: .env
    environment:
      - RAILS_ENV=production
      - APP_HOST=http://app:3000
    volumes:
      - ./storage:/rails/storage
    depends_on:
      app:
        condition: service_healthy
    restart: unless-stopped

4. Bring it up and register the first user

docker compose up -d

setup runs the migrations and exits. app brings up Rails, worker processes background jobs, and mail_fetcher polls IMAP every 30 seconds.

To register authorized senders, send an email from your ADMIN_EMAIL to the Frank FBI address with the subject add email1@exemplo.com, email2@exemplo.com. To list who’s registered, send with subject list. To see statistics, stats.

From there, any registered sender can forward suspicious emails and receive the analysis reports.

The development process

This project started as a simple idea: “what if I could forward suspicious emails somewhere and get an analysis back?” But security projects aren’t like normal projects.

In a regular project, a bug is an inconvenience. In a security project, a bug can be a vulnerability. If the system says a fraudulent email is legitimate, someone might lose money. If it incorrectly reports a legitimate domain as malicious, it can hurt an innocent company. If a malicious email manages to exploit the analyzer itself, the attacker gets access to the server.

That changes the development mindset. Every decision needs to consider: “how could a bad actor abuse this?”

Evolution through commits

Let me show how the project evolved by looking at the most relevant commits.

The project started with the basic email analysis structure and quickly needed access control:

eb59474 - Add admin access control and allowed senders whitelist
036e9f1 - Harden access control against email spoofing and admin impersonation

The first one adds the authorized sender list. The second solves the obvious problem: what if someone spoofs an authorized sender’s email? The system started verifying SPF/DKIM before accepting any submission. Per-sender rate limiting came along with that, to prevent abuse.

Then came concerns about scoring quality:

e891270 - Fix zero-dilution scoring bug and add critical alert UX
2cbb272 - Harden fraud scoring and reporting
50d96a1 - Move text pattern detection from regex to LLM consensus layer

The zero-dilution bug is subtle: when a layer fails and returns score 0 with low confidence, it would dilute the overall average downward, making fraudulent emails look safer than they were. The fix implemented dampening that discards low-quality layers instead of letting them drag the score down.

The shift from regex to LLM in pattern detection was pragmatic. Fraud patterns in natural language are hard to capture with regex. Too many false positives. LLMs understand context and intent in a way regex can’t.

Race conditions showed up when the pipeline started running layers in parallel:

2e4276d - Fix race conditions across pipeline and add Brave Search rate limiting
ac433d9 - Fix WHOIS race condition and Brave Search gzip error logging

Concurrent jobs trying to write to the same email record, WHOIS queries stepping on each other, external API rate limits getting blown. The classic “works in sequential tests, breaks in production.”

Preventing LLM hallucination got its own dedicated commit:

e08da9e - Prevent LLM hallucination in fraud reports

LLMs make things up. If the model says “this domain was registered yesterday” without evidence in the data, the report is compromised. This commit implemented cross-validation: claims from the LLMs are verified against the concrete data from the deterministic layers. If the LLM asserts something that contradicts the facts, the information is dropped or flagged.

And when community reporting was implemented:

a01d769 - Add community threat intelligence reporting
1b64eee - Harden community reporting against poisoning and add rate limiting

The first commit implements the feature. The second adds the anti-poisoning protections. If an attacker realizes Frank FBI reports automatically, they can try to send emails that contain IOCs from legitimate companies as “fraud,” making the system report innocent domains to community databases. That’s IOC poisoning, and it’s a real attack against threat intelligence systems.

Continuous hardening

9430c63 - Harden risky attachment analysis and warnings
bdc503d - Harden screenshot capture with failure recovery and pipeline timeout
125e73a - Separate suspect content from submitter signature in forwarded emails
6f7a522 - Handle forwarded message fidelity

Malicious attachments that could exploit the parser. Site screenshots that could lock up headless Chrome. Sender signatures that were being confused with the content of the suspicious email. Forwarded emails that lost fidelity in the forward. Each of these fixes a different attack vector or edge case.

Security isn’t a feature you implement once. It’s a continuous process of “what didn’t I think could go wrong?”

The numbers

860 lines per hour. Obviously AI-assisted development. But look at the hardening commits: none of them came from the LLM suggesting “hey, let’s protect against spoofing.” That was me stopping and thinking “wait, what if someone spoofs the authorized sender?”, “what if an attacker uses IOC poisoning against my reporting system?”. The LLM doesn’t ask those questions on its own. It implements them when you ask, but the one who has to spot the hole is you.

A serious warning: DO NOT offer this as a service

If you looked at Frank FBI and thought “cool, I’m going to offer this as a SaaS to other people,” I have one piece of advice: DON’T do that.

I can think of several ways to exploit a service like this offered to the public. The operator would have access to every email forwarded by users — personal information, corporate data, sensitive correspondence. A centralized service becomes a high-value target: compromise the server and you have access to a continuous stream of confidential emails from people who are already in a vulnerable situation.

I know enough people to know that whoever would deploy this as a service wouldn’t worry about encryption at rest or about destroying the emails after analysis. And who guarantees that the operator won’t read users’ emails? Nobody. It’s the kind of thing that looks like a useful service but in practice creates a honeypot of sensitive data managed by someone with no incentive to protect it.

Frank FBI was built to be self-hosted. To run on your server, under your control, with your data staying with you. Or in your company’s infrastructure, managed by your IT team.

And the project is licensed under AGPL-3.0. If you use my code and offer it as a network service, you’re legally required to release all the derived code. No exceptions. I picked AGPL for that reason — to make sure nobody takes the project, adds tracking and telemetry, and offers it as a “free email verification service” while collecting user data behind the scenes.

The repository is here. AGPL-3.0.

–

I have a huge collection of manga bought on Amazon.co.jp that I read through Kindle web. Shounen manga normally has furigana — that small text in hiragana next to the kanji — that helps me read, because, despite having studied Japanese, I was never formally trained. But manga aimed at adult audiences (seinen, not porn) usually comes without furigana. It’s pure kanji and my reading speed crawls.

For years I’ve wanted a tool that solves this. The idea is simple: detect the speech bubbles on a manga page, extract the text with OCR, and either add furigana to the kanji, or translate directly to English and render it back into the bubble. Sounds easy, right?

(kanji with no caption/furigana)

(with furigana injected in real time)

Yeah, it sounded easy. And because I knew it sounded easy but wouldn’t be, I never had the patience to actually do it. I know how to build the first 80% of any project. The problem is the last 20% — that phase of experimentation, tweaking, fine-tuning, handling edge cases — that consumes more time than all the rest combined. And in a computer vision project, that 20% is especially treacherous.

But then the vibe coding era arrived. And I thought: maybe now the 20% is feasible. I started the project on February 24, 2026, at 23:10 at night. And it became an example of how easy it is to produce massive volumes of useless code.

The original idea: OpenCV and heuristics

My original plan — the concept I’d imagined for years — was: use OpenCV — which is a famous and old computer vision library — to detect the speech bubbles. Manga bubbles are typically white areas with a black contour inside the panels. In theory, you just threshold the image to grab white regions, find contours, filter by size and shape, and you’re done.

In 24 hours I already had a working proof of concept: bubble detection, OCR with manga-ocr (a model trained specifically on Japanese manga text), furigana with MeCab for morphological analysis, translation with Ollama running Qwen3:14b locally, and rendering the text back into the bubble. The initial commit (9169d73) on Feb 24 already did all of that.

The next day, Feb 25, I was already extending the pipeline with a Korean webtoon translation flow. 27 commits that day. Everything seemed to be flowing.

And then the hell started.

The hell of false positives

The problem with OpenCV bubble detection is that manga isn’t a standardized document. Every artist has their own line, scan quality varies enormously between print eras, and color pages need completely different parameters from black-and-white pages. And the thing that looks the most like a white speech bubble on a manga page is… a face.

Character faces are light areas, relatively rounded, with a dark contour. Exactly like speech bubbles. And no matter how many filters you stack, there’s always a case where the face of a character with blue hair sails through every filter, or where a legitimate bubble gets rejected because it has an unusual shape.

Look at how my bubble detector ended up at peak complexity — 551 lines with 7 layers of false-positive filters (abridged version):

# --- False positive filters ---

# 1. Edge density
edge_pixels = cv2.countNonZero(cv2.bitwise_and(edges, edges, mask=mask))
edge_density = edge_pixels / area
if edge_density > max_edge_density:
    continue

# 2. Bright pixel ratio
bright_pixels = cv2.countNonZero(cv2.bitwise_and(bright_thresh, mask))
bright_ratio = bright_pixels / area
if bright_ratio < min_bright_ratio:
    continue

# 3. Mid-tone ratio
mid_mask = cv2.inRange(gray, 80, 220)
mid_pixels = cv2.countNonZero(cv2.bitwise_and(mid_mask, mask))
mid_ratio = mid_pixels / area
if mid_ratio > max_mid_ratio:
    continue

# 4. Contour circularity
circularity = 4 * np.pi * area / (perimeter * perimeter)
if circularity < 0.10:
    continue

# 5. Border darkness
border_mean = cv2.mean(gray, mask=border_only)[0]
if border_mean > 160:
    continue

# 6. Background uniformity (white_std)
white_pixels = gray[(mask > 0) & (gray > 200)]
if float(np.std(white_pixels)) > 15:
    continue

# 7. Dark content analysis (text strokes)
very_dark = np.sum((inner_mask > 0) & (gray < 60))
dark_ratio_60 = very_dark / inner_area
if dark_ratio_60 < min_dark_ratio:
    continue

Each one of those filters was added in response to a specific false positive. Edge density separated bubbles (which have sparse text strokes) from faces (which have hair, eyes, nose creating dense edges). Bright pixel ratio checked whether the region was actually white. Circularity discarded shapes that were too irregular. And so on.

But the worst part is that those filters interacted with each other in unpredictable ways. Look at this commit from Feb 26 (70c814a):

“Revert rect_dark, mid_ratio, and early-split changes that caused face FPs”

I had tried to relax two thresholds — rect_dark from 0.10 to 0.11, mid_ratio from 0.15 to 0.16 — to recover bubbles that were being missed. Result: faces and body regions started passing as false positives in Adachi manga. I had to revert everything.

That’s the pattern that repeated for days: recovering a missed bubble meant opening the door for false positives. Fixing a false positive meant losing a legitimate bubble. It was infinite whack-a-mole.

The band-aids: CLAHE, edge detection, watershed

When the 7 basic filters weren’t enough, I started stacking additional passes.

Commit 294e785 (Feb 26): added CLAHE (Contrast Limited Adaptive Histogram Equalization) as a second detection pass. Bubbles with mid-range brightness, near the 200 threshold, were being missed. CLAHE equalized the contrast and revealed those borderline bubbles.

But CLAHE also made faces pass through the filters because it artificially inflated skin brightness. So I had to add an entire validation function against the original image:

def _validate_on_original(candidate, gray_orig):
    """Check if a CLAHE-detected candidate looks bubble-like on original."""
    roi = gray_orig[y1:y2, x1:x2]
    mean_brightness = roi.mean()

    # Already bright enough for pass 1 — rejected for good reason
    if mean_brightness > 215:
        return False

    # Must have text strokes (dark pixels)
    dark_ratio = np.sum(roi < 60) / roi.size
    if dark_ratio < 0.07:
        return False

    # White pixel variance: text creates high std, face skin is uniform
    white_pixels = roi[roi > 200]
    if len(white_pixels) > 50:
        if float(np.std(white_pixels)) < 9:
            return False

    return True

Commit 5dddb31 (March 1): added an entire third detection pass based on edges (edge-based segmentation). To catch bubbles where the white interior blended into the white background of the page. Dilated the Canny edges, inverted, did an AND with bright regions, and looked for contours in the result.

Commit b695295 (Feb 26): added recovery of small bubbles via morphological gradient + OCR validation. If OCR confirmed there was valid Japanese text in the region, it was probably a real bubble.

Each band-aid added 50-100 lines of code and another layer of complexity. And each one had its own edge cases and false positives.

The final tally for v0.1

On March 1, I tagged v0.1. At that point I had:

• 90 commits in 6 days of development (Feb 24 to March 1)
• 551 lines in bubble_detector.py alone
• 7 layers of false-positive filters, each with empirical thresholds
• Two complete detection passes (original + CLAHE)
• A third edge-based pass
• Cross-validation against the original image
• Watershed separation for overlapping bubbles
• Separate threshold profiles for color vs black-and-white pages
• 20+ magic numbers tuned empirically against a limited sample of pages
• Regression tests pinning each specific false positive (face of a girl with blue hair, window frame, thin horizontal strip, concrete floor…)

And even with all that, it still wasn’t reliable. Every new manga I tested revealed some little case that broke the detector. It was an extremely brittle monolith.

SYMPTOM: you’re patching a fix that fixed another fix that was fixing yet another fix, and when you touch one piece, you accidentally break another: that means the code is brittle, a house of cards about to collapse. That’s the moment to give up and rethink!

The decision: research alternatives

At v0.1 I stopped and asked the question I should have asked at the start: is there someone who already trained an ML model to do exactly this?

I asked Claude to research available models for comic bubble detection. The research produced the document docs/yolo_bubble_detection_plan.md, where we analyzed the alternatives.

The first one that came up was a YOLOv8 Medium from ogkalu (comic-speech-bubble-detector-yolov8m), trained on ~8,000 images of manga, webtoon, manhua and Western comics. It detects only one class (speech bubble). But digging deeper, we found another model from the same author: ogkalu/comic-text-and-bubble-detector, an RT-DETR-v2 with a ResNet-50-vd backbone (42.9M parameters), trained on ~11,000 images, with three classes: bubble, text_bubble and text_free. Both Apache 2.0.

We also evaluated the comic-text-detector (DBNet + YOLOv5, ~13,000 images from Manga109), but that one detected text regions and not bubbles. And as future training data, there was Roboflow with 4,492 already-labeled images, and the Manga109 dataset with 147,918 annotations across 21,142 pages.

The RT-DETR-v2 with 3 classes was the most promising because it distinguished speech bubbles, text inside bubbles and free text (narration, SFX). It could replace both bubble_detector.py and text_detector.py in a single inference pass.

The conclusion of the research document was direct:

“Even without fine-tuning, these models were trained on 8 to 11 thousand images of diverse comics. They should handle the artistic style diversity that our manual filters struggle with. The 7-filter heuristic cascade and its magic numbers would be completely eliminated.”

And if it weren’t enough, we included a fine-tuning plan using paired data (original Japanese pages vs English fan translations) that generated labels automatically through image diff. But first we wanted to test the baseline.

Replacing everything: RT-DETR-v2

On March 4, I made commit 0df63f2: “Replace OpenCV heuristic detection with RT-DETR-v2, add bubble shape masking”.

The diff speaks for itself:

16 files changed, 732 insertions(+), 1112 deletions(-)

1,112 lines deleted. More lines removed than added. The 551-line bubble detector was replaced by 262 lines — and most of those are shape mask extraction (contour mask) from the detected bbox, not the detection itself.

The detection core became this:

MODEL_ID = "ogkalu/comic-text-and-bubble-detector"
DEFAULT_CONFIDENCE = 0.35

def detect_bubbles(img_cv, confidence=DEFAULT_CONFIDENCE):
    model, processor, device = _get_model()
    img_pil = Image.fromarray(img_cv[:, :, ::-1])  # BGR→RGB
    inputs = processor(images=img_pil, return_tensors="pt").to(device)

    with torch.no_grad():
        outputs = model(**inputs)

    results = processor.post_process_object_detection(
        outputs, target_sizes=target_sizes, threshold=confidence
    )
    # ... map classes, deduplicate, sort

No magic thresholds. No separate profiles for color vs B&W. No CLAHE. No edge detection. No watershed. No 7 filter layers. A model trained on 11,000 diverse comic images already knows how to tell a bubble from a face better than my pile of “ifs.”

What also vanished alongside it:

• The entire text_detector.py (387 lines) — replaced by RT-DETR’s text_free class
• The false-positive feedback system — RT-DETR detections are reliable enough to not need manual marking
• Hundreds of lines of regression tests pinning specific false positives

In the end, I didn’t even need to try fine-tuning my own model. The model that already existed solved more than 99% of the cases, which for me was already excellent.

People who don’t understand statistics struggle with this. With my “manual” OpenCV procedure I was already getting 80%, maybe more. But that’s very little. If on every page a face gets a bubble slapped on top of it, that’s terrible.

Even if with a lot of effort (another five hundred different “ifs”) I could get to 95%, that’s still not enough. Reaching 80% is easy. The last 20% costs exponentially more, and the last 1% can be impossible in many cases. That’s how things work. Everyone stops at 80%.

The other pains: Flutter on Linux

While the Python and Go backend was relatively stable, the Flutter client on Linux was a separate chapter of suffering.

Flutter uses WebKitGTK for the WebView on Linux, and that component has painful particularities. Commit 8e5d168 (Feb 28) tells the story: WebKitGTK can’t resolve Promise return values from asynchronous JavaScript, generating a PlatformException. I had to rewrite all the overlays as synchronous IIFEs with decode().then() and an opacity: 0.999 nudge to force a texture refresh in the GPU compositor.

On my NVIDIA + Wayland setup, the WebView was unusable at full resolution. Commit a36e1eb (Feb 27) tried to fix it by forcing CPU rendering with WEBKIT_SKIA_ENABLE_CPU_RENDERING=1 and disabling accelerated compositing. Then I had to revert that (commit 8e5d168) and force the AMD iGPU’s Mesa via __EGL_VENDOR_LIBRARY_FILENAMES to stop WebKitGTK from grabbing the NVIDIA dGPU.

Each one of those discoveries cost hours of debugging things that simply had no documentation. In the end I got the Linux version to a reasonable state, but it’s nothing spectacular. I don’t know if I’m missing something obvious, but Flutter on Linux I found to be a drag, especially right after building a native app with Rust/Tauri (much better). But since I wanted an app that worked on Linux and Android, there weren’t that many options.

The Kindle prefetch that died

An idea that seemed brilliant and went wrong: create a second hidden WebView in Flutter that shared cookies/session with the main WebView and kept navigating pages ahead to pre-process them.

The reason is that the Kindle website only loads one page at a time. It doesn’t load everything at once or in chunks, just one page. So you can’t process pages forward or backward. When the server was still slow at processing pages, I wanted to keep everything pre-cached, so when the user turned the page the translation was already there.

Commit a36e1eb (Feb 27) implemented the entire KindlePrefetchManager: 406 lines of Dart, with batched prefetch (3 pages ahead), trusted GDK events for turning the page (isTrusted=true), rate limiting with human pacing, a window with sensitive=FALSE so it wouldn’t steal focus.

In the following days, the fixes piled up:

• cfe08eb: improve prefetch reliability and overlay matching
• 4461b5f: harden overlay selection and Kindle recovery
• f141b2e: stop destroying the background webview on every page turn

And in the end (commit 2b93e99, March 4), I deleted everything. 657 lines removed, replaced by a simple spinner in the toolbar.

With the move to the RT-DETR model, and parallelizing the bubble translation, the translation became “almost real-time,” loading in less than 10 seconds, so it’s no longer such a big deal to wait for a page to come in, and you can ask one at a time. The prefetch added too much complexity for marginal gain, and the right way to solve it was simply to process the page on demand with proper visual feedback.

The numbers

In total, the Frank Yomik project today has:

• 111 commits across 9 days of development (with a 2-day break in the middle)
• 29,428 lines of code in 181 files
• ~13K lines of Python (processing pipeline + worker)
• ~6K lines of Dart (Flutter client)
• ~3.4K lines of Go (API server)
• 345 unit tests + 34 integration tests
• Support for Japanese manga (Kindle) and Korean webtoons (Naver/Webtoons)

The pipeline today works like this:

1. The Flutter client (Android or Linux desktop) opens the Kindle or Webtoons site in a WebView
2. Captures the page image (capturing the image blob on the Kindle page, fetching the <img> for webtoons)
3. Sends it to the Go API which queues it on Redis Streams with SHA256-based dedup
4. Python worker processes: RT-DETR-v2 detects bubbles → manga-ocr or EasyOCR extracts text → Ollama qwen3:14b translates → text_renderer renders back into the image
5. Result comes back over WebSocket, the overlay replaces the original image

For furigana: fugashi (a MeCab wrapper) does morphological analysis and generates the hiragana reading for each kanji. I switched from pykakasi to fugashi because pykakasi doesn’t consider sentence context (「人」 became にん instead of ひと).

If I knew then what I know now — that the RT-DETR-v2 model already existed and solved the detection problem with a confidence threshold — I would have eliminated the entire OpenCV phase. The OCR, translation, rendering and Flutter parts were already reasonably stable. Detection was the bottleneck, and it was exactly the part I could have saved myself, if I’d given up sooner.

What I learned by failing

I spent 5 days polishing an OpenCV bubble detector that had 20+ tuning commits, 7 filter layers, 3 detection passes, and that in the end was replaced by a 50-line wrapper around a pretrained model.

It was 1,112 lines deleted in a single commit. But it wasn’t wasted time. Those 5 days taught me exactly why heuristics fail in computer vision. I understood the problem, the cascade where touching one threshold breaks another, and it was that understanding that made me recognize when to stop and research alternatives.

And here’s where the real role of vibe coding in this story comes in.

Prompting doesn’t replace thinking.

I generated code very fast with Claude, but the problem wasn’t writing speed, it was the approach. No prompt in the world turns 7 layers of heuristic filters into a robust solution. The right solution was to change the approach completely.

But vibe coding made the failure cheap. In the pre-AI days, those 5 days of OpenCV would have been maybe 2-3 weeks. The cost of being wrong would have been high enough to make the decision to throw it out really hard. With vibe coding, 5 days were discarded without pain because I knew I could rebuild fast. And in fact, the RT-DETR-v2 integration and the restructuring of the entire project were done in a single day (March 4, 15 commits).

The question that remains: if I had done the yolo_bubble_detection_plan.md research on day 1 instead of day 5, I’d probably have reached the current state in 2 days. The difference between weeks of work and a weekend was a HuggingFace search. Researching before implementing seems obvious in hindsight, but in the heat of the moment the temptation to solve it by hand is strong.

The project is now open source. I initially didn’t know if I wanted to open the code, there was a lot of kludge and that 551-line detector that I was embarrassed about. But after the refactoring, the code became clean enough to share. It’s the version I would have liked to have built from the start, but that I could only build because I screwed up first.

The bigger headache was the bubble detection and replacement model, but there are several other points I didn’t detail: did you notice that webtoons are colored and the bubbles themselves have art? I had to use an image model to do in-painting and have the AI redraw the bubble before placing the text on top.

Another headache I don’t think I’ll solve: translation coherence. Today it translates each bubble in isolation, with no context of the story before or after. In Japanese, there’s no gender distinction in words. So the captain was talking about Nami, but the bubble says “He” instead of “She.” There’s no way to know without reading the previous text. For this to work better, like in a GPT chat, you have to add part of the previous text to know when to use the correct gender or, worse, when there are puns that appeared volumes ago and are referenced in the future (something an Oda loves to do). All those subtleties get lost if you only translate one bubble at a time.

I imagine that’s why nobody has done something like this yet, translating in near real time, because for it to really be good, the translation work would be exponential for each chapter further along in the story.

Anyway, the name Frank Yomik comes from “yomi” (読み, reading in Japanese) and “ik-da” (읽다, to read in Korean). Frank is a nod to a frank, direct translation. The app reads in both languages.

For anyone who wants to try it: the repository is on GitHub. The server needs a GPU with at least 8GB of VRAM for the detection model + OCR + translation. The Flutter client runs on Android and Linux desktop. And if you, like me, have a stack of Japanese manga you’d like to read more fluently — now you can.

–

I published the “final post” of the Behind The M.Akita Chronicles series 10 days ago. 274 commits, 1,323 tests, deployed to production. I wrote down the lessons, did the conclusion, dropped the quote at the end. Done.

125 post-production commits later, I can confirm: software “done” doesn’t exist.

Today the M.Akita Chronicles repo has 335 commits and 1,422 tests. Tomorrow, Monday, subscribers get the 3rd consecutive newsletter — generated, reviewed, and sent by a system that won’t stop evolving. Meanwhile, Frank Sherlock went from 50 commits and v0.1 to 103 commits and v0.7 with face detection. FrankMD got 3 external contributors and shipped v0.2.0. And even FrankMega — a project built in 1 day — needed fixes when real users showed up.

Throughout February, I published more than a dozen posts detailing the build of each project. This post is different. It isn’t about building from scratch, I covered that already. It’s about what happens after. And what happens after destroys the one-shot prompt narrative. Software needs an experienced human at the wheel. Iterative development is the only thing that works. Anyone who disagrees hasn’t put a system in production yet.

I’ll prove it with git log.

Life After Deploy

Let’s get into the details.

The M.Akita Chronicles: 3 Weeks in Production

The newsletter system has been live since February 16. It’s already sent 2 newsletters and tomorrow it sends the third. 56 commits have happened since the “final post”:

Features Nobody Saw Coming

Steam Gaming. Wasn’t in the original plan. It was born because the entertainment section only had anime — and gamer readers were left out. Result: 6 commits just on launch day. The first commit (bf41e25) added the whole section — Steam API service, generation job, tests, Hugo shortcode. The next 5 fixed things: Portuguese date parsing, ranking offset so the top 10 didn’t repeat, wishlist filter for releases, dark theme. No spec in the world predicts that the Steam API returns dates with Portuguese abbreviations ("fev", "mar", "abr") when you ask for l=brazilian. You find that out when the parser breaks in production.

Preview system. /preview was born because I needed to see how the auto-generated content looked before publishing the entire newsletter. Seems obvious in retrospect, but in v1 the only way to validate was to generate everything and look at the final result. 5 commits to build a preview system with 9 sections, aliases (hn for hacker_news, steam for steam_gaming), separate comment-preview mode, and rescue when one section fails without taking the others down (d23c725 — because a section with an error was killing the entire preview).

Marvin’s moods. Marvin (the bot’s sarcastic persona) suffers from a chronic LLM problem: smoothing. Without intervention, every comment ends up the same lukewarm tone. The iterative fix was a mood system — 9 modes the operator picks per story:

MARVIN_MOODS = {
  "sarcastic"   => "Be extra sarcastic and biting in your commentary.",
  "ironic"      => "Use irony and dark humor to make your point.",
  "grounded"    => "Be more neutral and journalist-like, factual and measured.",
  "provocative" => "Be provocative and controversial, challenge assumptions.",
  "counter"     => "Take the opposite position from Akita's comment.",
  "insightful"  => "Find a non-obvious angle -- a historical parallel, a second-order consequence.",
  "positive"    => "Find the genuinely positive angle.",
  "hopeful"     => "Be cautiously optimistic. Acknowledge the good potential.",
  "negative"    => "Be extra pessimistic and bleak."
}.freeze

None of this came from a spec. It came from 3 weeks of reading bland comments and thinking “this isn’t good enough”. There are 7 commits just for prompt tuning across 10 days. I banned formulaic patterns (d4bd3a6). I killed off “Ah,” as an opener (4171628). I removed Marvin from entire podcast sections because he was contaminating the tone (246cc60). I pushed for substance instead of puns (8bc47c4). Each prompt commit is a micro-correction that only makes sense after reading the previous output.

Bugs That Only Exist in Production

Of the 56 commits, 13 are bug fixes. Some favorites:

• Gmail clipping (3258f5b): Gmail silently cuts off emails larger than 102KB. I found out when the 2nd newsletter went out longer and Gmail readers didn’t see the end. Fix: a size check in the content preflight that warns before publishing. I shrunk the history sections from 10 to 5 items (51ce528) to fit the limit.

• False paywall detection (1fd8739): the scraper marked sites as paywalled when it found accidental block text in a generic footer. Only showed up when the story list grew to dozens of real sources.

• t.co link mangling (6bd056f): Twitter shortens URLs with t.co, but it also auto-links file names that look like domains. config.yml becomes a link to the config.yml domain. 158 lines of fix to handle tweet text correctly.

• TTS language revert (7803ad9 -> c1dd668): I tried switching the TTS language from “Portuguese” to “Auto”. The model produced a mixed accent. Reverted the same day.

• Podcast section ordering (5bef18c): podcast sections came out in the wrong order. 3-line fix + untracking of 1,504 lines of generated content that had been committed by accident.

• Elementor sites with multiple <article> (e70b756): sites built with Elementor use the <article> tag in a non-semantic way. The content extractor was grabbing the wrong block. 171 lines of fix + tests.

None of these bugs could have been predicted in a spec. They only exist because the system is live, processing real data, hitting real APIs, being read by real people.

From 1,323 to 1,422 Tests

The test suite kept growing along with it:

New feature? Comes with a test. Bug fix? Comes with a regression test. Without that, the 56 post-production commits would be 56 chances to break something that worked yesterday. TDD isn’t a phase, it’s a habit.

The Model Changed: From Claude to Grok-4

Commit 8f9d11c: Switch default model to x-ai/grok-4. The architecture had isolated the LLM model into an environment variable from day one. But the model name was still hardcoded across 24 files. Result: commit c8c688e — Centralize LLM model config — 24 files touched to make swapping models a 1-line config change.

This kind of refactoring only shows up in operation. When you start the project, you don’t know you’re going to change models. When you change, you want it to be trivial. And the cleanup to make it trivial? Only happens when the pain shows up.

Frank Sherlock: From v0.1 to v0.7 in 4 Days

The Frank Sherlock post covered the first 2 days and ~50 commits: the benchmark research, the Tauri app scaffold (Rust + React), the classification pipeline with Ollama, and the v0.1.0 release with binaries for Linux, macOS and Windows.

What the post didn’t cover: the next 53 commits, in 4 days, that took the project from v0.3 to v0.7:

Face Detection: A Feature Born from Iteration

v0.1 was an image indexer with LLM-based classification. v0.7 has face detection with 512-dimensional embeddings, cosine-similarity clustering, and per-person search (face:alice). None of that was in the original plan. It came up because I started using the app with real photos and thought “I want to find every photo of so-and-so”. The feature was born from use, not from a spec.

The path to get there was methodical:

1. f34132f: A/B testing framework for face detection — benchmark before implementing, not after
2. ef3be82: A/B results (SCRFD + ArcFace won)
3. 6d9174a: Native implementation with ONNX Runtime — no Python, no external dependency
4. 3b25eaf: Clustering, person management, complete FacesView
5. a02d67f: Refactoring — extract helpers, delete dead code, share CSS

Benchmark -> implement -> refactor. The same cycle as always. And something no one-shot prompt produces.

Video Support: Another Emergent Feature

The app was built for images. But real media folders have videos. Commit a67a2f9 added full support: scanning of 11 formats (MP4, MKV, AVI, MOV, WebM…), keyframe extraction for LLM classification, black frame skipping, .srt subtitle parsing for the full-text index, and inline preview with HTTP Range streaming.

1,626 lines added in one commit. An entire feature born from real use, not from a spec.

7 Releases, 3 Platforms, Automatic AUR

In 7 days, Frank Sherlock had 7 releases. Each one with binaries compiled for Linux (AppImage), macOS (DMG with notarization and code signing), and Windows (MSI). The CI/CD pipeline (release.yml) does everything automatically. Starting at v0.5.0, an additional workflow (aur-publish.yml) publishes automatically to the AUR — the Arch Linux package repository.

Final tally: 17,210 lines of Rust, 10,863 lines of TypeScript, 621 tests (322 Rust + 299 frontend). Cross-platform. Releases published and installable by any user. One week, one dev, one AI agent.

The CI/CD cross-platform part is where the “works on my machine” fantasy dies. There were 17 commits just for builds: Windows UNC paths, macOS signing with notarization, Linux AppImage, release workflow permissions, macOS Intel target removed. No LLM in the world knows that macOS requires specific entitlements for hardened runtime, or that paths on Windows start with \\?\. That kind of deploy work demands someone who’s been through that pain before.

FrankMD: Real Open Source

FrankMD was the first project in this saga, a self-hosted Markdown editor with Rails 8. The February posts covered the build. What happened after is more interesting: other people started contributing.

14 commits since February 20. 3 external contributors. 4 PRs merged. v0.2.0 release on the 28th:

The Maintainer’s Pattern

What’s most telling isn’t the contributions themselves, but what I did after every merge.

When @LuccaRomanelli submitted the Omarchy theme sync PR (+308 lines), I merged it and immediately committed fix: harden Omarchy theme integration and fix broken tests — +163 lines of fixes and tests in the next commit. The contributor implemented the feature. The maintainer hardened it.

When @rafaself submitted the table hint fix, there were 3 iterative commits the same day — enhance, improve, streamline — showing the same progressive refinement pattern I do with the AI agent. The next day, he sent a separate commit updating faraday, nokogiri and rack for security advisories.

On the 28th, I sat down and did everything in one shot: merged the 4 PRs, committed the hardening, updated 5 gems (brakeman, bootsnap, selenium-webdriver, web-console, mocha), and published the v0.2.0 release notes. A classic “release day”.

FrankMD today has 226 commits, 1,804 tests (425 Ruby + 1,379 JavaScript), and active contributors. It isn’t “my little personal project” anymore. It became software with a community. And community doesn’t show up for a test-less prototype that “works on my machine”. It shows up for a project with green CI, documentation, versioned releases, and code you can actually read.

FrankMega: Even the Smallest Project Needs Post-Production

FrankMega was built in 1 day. 26 commits, secure file sharing with Rails 8, 210 tests, deploy via Docker + Cloudflare Tunnel. Post published the same day. Done, right?

Three days later, 2 commits:

• db4bb705 — Add macOS, Linux, and Windows package MIME types to seed defaults
• b0c4829a — Fix MIME types to match Marcel detection, add normalizes strip

Users tried to share .dmg, .deb, and .msi files. The MIME type detection (via the Marcel gem) didn’t recognize those formats because they weren’t in the seed defaults. Two commits. 22 lines. 15 minutes.

But without them, FrankMega couldn’t be used to share installer packages — which is exactly the most common use case on a dev’s home server.

No prompt in the world predicts that your users will want to share .deb files on day one.

The simplest project of all, with the shortest post-production. And even then, it needed iteration.

The Consolidated Numbers

692 commits. 4,057 tests. 4 projects in production. February 2026. Me and an AI agent.

Why One-Shot Prompting Is a Myth

Look at the 125 post-production commits and tell me which one of them could have been predicted by a spec.

Gmail cuts off emails larger than 102KB, and you only find that out when you send the first long email. The Steam API returns dates with Portuguese abbreviations ("fev", "abr"), and you only find that out when the parser breaks on generation Sunday. macOS users try to share .dmg files and can’t because the MIME type isn’t in the seed. Marvin opens every comment with “Ah,” and you only notice it after reading 30 in a row and wanting to claw your eyes out. The LLM model has to be swappable with 1 line of config, but it’s spread across 24 files. Windows uses UNC paths starting with \\?\, and CI explodes in your face. And TTS in “Auto” mode? Generates an accent that sounds like Lisbon Portuguese trying to be carioca.

None of this is “debugging” in the traditional sense. It’s navigating a problem space that only reveals itself when the software meets reality. Each one of these was a real-time decision made by someone with context.

The one-shot prompt fantasy is that, if you write a sufficiently detailed spec, the AI produces the perfect software. But the perfect spec would require you to know in advance everything that’s going to go wrong. And if you knew in advance everything that’s going to go wrong, you wouldn’t need the spec — you’d already have the software done in your head.

Good software is the result of hundreds of micro-decisions made with the system running. Not a single macro-decision made before the first line is written.

The Role of the Experienced Developer

AI accelerated all of this. Without it, 692 commits in a month would be impossible for one person. But acceleration without direction is just faster entropy.

In every project there were decisions the AI couldn’t have made on its own. Switching from Claude to Grok-4 because the previous model was weak in a specific domain. Benchmarking SCRFD against alternatives before implementing face detection, because the wrong choice would cost days. The +163 lines of hardening I committed immediately after merging the Omarchy PR, because I saw where it was going to break. The TTS revert from Auto mode the same day, because I know that “Auto” on a TTS model for Brazilian Portuguese will produce a mixed accent.

The circuit breaker case is the most illustrative. When I added rate limiting, the Brave Search API started returning 429s every once in a while. If I had asked, the agent would have implemented a retry with exponential backoff. But I didn’t ask for retry. I asked for a circuit breaker:

class WebSearcher
  CIRCUIT_BREAKER_SECONDS = 120

  def self.search(query, max_results: 5)
    return [] if circuit_open?

    response = brave_search(query, max_results)

    if response.code == 429
      trip_circuit!
      Rails.logger.warn("WebSearcher rate limited (429)")
      return []
    end
  rescue Net::OpenTimeout, Net::ReadTimeout => e
    trip_circuit!
    []
  end
end

If the API returns 429 or times out, it stops trying for 120 seconds. No retry, no backoff, no queue. Why? Because I know the cron runs every day at 8am and that on newsletter generation Sundays the query volume triples. A herd of retries delaying everything is worse than empty results in one section.

The AI implements a circuit breaker when you ask it to. But it isn’t going to ask to implement a circuit breaker. It doesn’t have the operational context. That’s knowledge that comes from running systems in production for decades.

The agent writes the code. I decide what code to write. And that decision requires experience no prompt can replace.

The Lessons

1. Software in production diverges from the spec in hours, not months. The first M.Akita Chronicles bugs showed up in the first real newsletter. FrankMega’s MIME types broke on the first uploads. Anyone who thinks deploy means done is living in a world that doesn’t exist.

2. Post-production isn’t “maintenance”. 56 commits in 10 days on the M.Akita Chronicles aren’t patches. They’re new features (Steam Gaming, preview, moods), architecture refactoring (LLM config centralization), security hardening (rate limiting). That’s development. The software didn’t stop evolving just because I said “done” in a blog post.

3. TDD protects evolution. 99 new tests in 10 days. FrankMD’s 1,804 tests let me merge 4 external contributor PRs without fear. Without tests, every merge is Russian roulette.

4. Small releases keep you sane. 7 Frank Sherlock releases in 7 days. Green CI, compiled binaries, release notes. Something broke? Roll back one version. Compare that with “6 months + big bang release” and tell me which one works better.

5. Community shows up for real projects. Nobody sends a PR to a test-less prototype that “works on my machine”. FrankMD got 3 contributors because it has green CI, documentation, and versioned releases.

6. The developer’s experience is the bottleneck, not the AI’s speed. 692 commits in a month. But every commit that mattered required decades of experience to know it was needed. The AI types fast. I know what to type.

7. One-shot is for demos. Iteration is for production. If the goal is a 10-minute video showing a “finished SaaS”, one-shot will do. If the goal is software that survives contact with real users, only iteration works. And sustainable iteration demands discipline: TDD, CI, small releases, continuous refactoring. No shortcut.

To the Senior Still Sitting on Their Hands

So far I’ve been hitting the amateurs who think they can fire off a prompt and the AI spits out a finished system. Fair enough. But there’s another group that worries me just as much: the senior developer who saw the AI mess up three times, declared “this is useless”, and went back to doing everything by hand.

I get the reasoning. The AI hallucinates, generates code with subtle bugs, suggests over-engineered solutions. All true, and I documented every one of these problems in the previous posts. But tell me something: have you never done exactly the same thing? Have you never spent 2 hours reading documentation only to find out it was a typo in the config? The difference is that when the AI gets it wrong, you catch it in the tests and fix it in minutes. When you get it wrong on your own, it takes you the same amount of time to make the mistake and even longer to find it, because you trust your own code.

Let’s look at the concrete numbers of what I shipped in February, one person and one AI agent:

Now do the math in your head. How long would it take you to build Frank Sherlock on your own? A Tauri app from scratch, with an LLM classification pipeline in Rust, OCR via Python, full-text search with FTS5, duplicate detection by perceptual hash, face detection with native ONNX, video support with keyframe extraction, CI for 3 platforms with macOS code signing and notarization, auto-update, and automatic publishing to the AUR. With 621 tests. In Rust, which doesn’t forgive.

To be honest: without AI, a good senior dev would take at least 3-4 weeks on this, probably more. I did it in 7 days, alone, and every release is published with binaries anyone can download and install.

I already estimated The M.Akita Chronicles in the previous post: ~200 user stories. In Scrum with a senior team of 2-3 devs, no impediments, that would be 10-15 weeks. I did it in 8 days. Today, 3 weeks later, the system keeps running, evolving, with 99 more tests than when it “was done”.

FrankMega is more modest, but secure file sharing with Rails 8, I18n in 2 languages, 22 security issues fixed, Docker deploy, 210 tests. I did it in 1 day. A good senior, without AI, would do it in 1-2 weeks at best.

And I’m not talking about disposable prototypes. People from outside send PRs to FrankMD. Real subscribers get the M.Akita Chronicles newsletter every Monday. Anyone can download Frank Sherlock from GitHub Releases or the AUR. Green CI, Brakeman clean, real tests, automated deploys. That’s production software, not a conference demo.

“Yeah, but I write better code without AI.” Maybe. But how long does it take you? What I showed here isn’t that AI writes perfect code. Far from it. It’s that with TDD, CI, pair programming with the agent, and continuous refactoring, the end result is production code with quality. 4,057 tests are there to prove it. Brakeman clean. 125 post-production commits show the code can take evolution without turning into a ball of mud.

And using AI here and there to generate a snippet of code, like glorified autocomplete, also isn’t the answer. You’re leaving 90% of the gain on the table. What I did in February was full-time pair programming with an agent. From the first commit to production deploy. With the same discipline I’d use with a human pair. Result: 4 projects in production in 1 month, with quality I put my name on. Because I did put my name on it.

If you’re a senior and you’re still waiting for AI to “get better” before you really start using it, here’s my message: it’s already good enough. The 692 commits are there to prove it. The bottleneck now is you learning to work with it.

Conclusion

In February 2026, I built 4 projects from zero to production with AI. But the build is the easy part. What separates real software from a demo is the 125 commits that came after, when the bugs nobody predicted appeared, when external contributors sent PRs that needed hardening, when new features came up from real use and not from a requirements spreadsheet.

AI made me absurdly more productive. Without it, Frank Sherlock wouldn’t have face detection in 7 days. Without it, M.Akita Chronicles wouldn’t be in its 3rd week of operation with 1,422 tests. The speed is real.

But none of the decisions that mattered came from the AI. Switching models, reverting the TTS the same day, looking at the contributor PR and seeing where it was going to break, asking for a circuit breaker instead of retry. All of that was me. The AI executed. The decisions were mine.

The AI is the accelerator. Extreme Programming techniques (TDD, small releases, pair programming, continuous refactoring) are the brake and the steering wheel. Without discipline, AI produces fast code that piles up technical debt even faster. With discipline, AI produces software that actually evolves, week after week.

692 commits. 4,057 tests. 4 projects in production. And tomorrow, Monday, at 7am, M.Akita Chronicles subscribers get the 3rd newsletter. Generated, reviewed, and sent by a system that will never be “done”. Because finished software is dead software.

“Finished software is dead software. Live software iterates.”

The name is Frank Sherlock — a local image cataloging and search system using AI. You point it at a folder (it can be a NAS with terabytes), it scans everything, classifies each file using a vision LLM running locally on your GPU, and gives you full-text search over the content. It’s not cloud, nothing is sent out, it runs 100% on your machine.

Here are some examples of text it extracted from some of my images:

And check out the Surya OCR details: it read the text on the Game Boy screen perfectly:

More than that, I have directories of screenshots of payment receipts. I would never find anything in there again, but now:

It does make some mistakes, of course, with obscure anime (it keeps thinking everything it doesn’t recognize is Evangelion 😂). But it surprisingly gets many of them right too. And the descriptions themselves already help a lot.

You’ll need a minimum GPU (I only tested on my 5090, but there are smaller models to download for smaller GPUs, and it theoretically supports AMD and Mac too, but I haven’t tested yet - I’ll accept Issues and Pull Requests from anyone who wants to do beta-testing on Mac and Windows). You just need Ollama installed and running; optionally, Python (to have Surya OCR, which is optional, but is the best).

I’ll mark it as “1.0” when I have more people testing on Windows/macOS and I have the certificates to sign the executables properly. This is still a “beta” version! Compile and run it yourselves on your machines, everything is explained in the README.

And I did this with my “Agile Vibe Coding” — basically, programming in partnership with an LLM (in this case, Claude Code).

Agile Vibe Coding works. And it works very well. But the idea is only 10% of the work. The other 90% is engineering.

Engineering requires experience, judgment, and knowing how to ask the right questions. The LLM is an excellent executor. But whoever decides what to execute, in what order, and why, that’s still the developer’s job.

There’s a growing discourse that anyone with a good idea can build software now. In a certain sense, yes, you can get a prototype up fast. But the distance between “runs on my machine” and “software that works on 3 operating systems, survives cancellations in the middle of processing, doesn’t corrupt data, and scales from 94 test files to 500,000 in production” is still enormous. That gap is engineering, and engineering still demands someone who knows what they’re doing.

I’ll tell the complete story: from initial research to release, going through benchmarks, proof of concept, architecture decisions, multi-platform CI/CD, and everything that sat between “I have an idea” and “here’s the AppImage, the DMG, and the MSI”.

The Original Idea

It all started with a simple question: can open-source vision LLMs actually classify content? I’m not talking about “woman on the beach” — any model does that. I’m talking about looking at an image and saying “this is Ranma, from the anime Ranma 1/2 by Rumiko Takahashi, in a scene from the OVA The Battle for Miss Beachside”. Are we at that level yet? (TL;DR no, but enough)

And if we are, can you build a smart file catalog? Something where I point at my NAS with terabytes of media accumulated over decades and can search by content, not by filename? Anyone with a home NAS knows: after a few years, files pile up and directory organization simply stops scaling. You know you have that 2019 payment receipt somewhere, but the file is called IMG_20190315_142301.jpg and it’s in a directory with 3,000 other photos.

My hardware: AMD 7850X3D, RTX 5090, Arch Linux. Absolute restriction: no remote APIs, no OpenAI, no cloud. Everything open-source, everything local. If I’m going to process terabytes of personal files, including financial documents and private photos, I don’t want to send anything to third-party servers. Plus the cloud API cost for that volume would be prohibitive.

But first: research. Without knowing if the technology delivers what I need, there’s no point building anything. Small scripts, quick prototypes, different models. See what works before writing the first line of code of the real app. This is a pattern I’ve followed for years: validate the riskiest assumption first. If the vision LLM can’t classify decently, everything else is a waste.

A/B Research: Benchmark Driven Development

This is the part most people skip, and it’s exactly where experience makes the difference. The temptation to jump straight into implementation is enormous. “I’ll use model X because I read on a blog that it’s good.” No. Before choosing model, framework, or architecture, I set up a formal benchmark.

I built a test corpus with 94 files: 60 images (photos, screenshots, anime, documents, receipts), 9 audios, 13 videos, and 12 documents. For each file, I created a ground truth in JSON with the correct classification — type, description, series (when applicable). That ground truth is what lets you measure real accuracy, not “I looked at the result and it seemed OK”.

The benchmark has 6 phases, each answering a specific question:

1. Metadata: how much does it cost to extract basic metadata? (answer: cheap, 0.07s/file)
2. Images: which vision model is best? Which OCR?
3. Audio: does Whisper work? Which model size?
4. Video: does frame-based classification work?
5. Unified catalog: does the full integrated pipeline work?
6. Cost projection: how much time and money to process a real NAS?

Phase 2: Vision — The Result Nobody Expected

I tested qwen2.5vl:7b, llava:13b, and minicpm-v:8b on 30 labeled images. The result:

The 7B parameter model crushed the larger ones. It’s not a typo. qwen2.5vl:7b beat llava:13b (almost twice its size) in every metric, and was also 3x faster. This contradicts the intuition of “bigger model = better model”. In practice, it depends on the task and the prompt.

Naturally, the next question is: what about the 32B? Same model, giant version. We should be able to get much more, right? Wrong:

The 32B gave +0.06 in series accuracy (literally 1 more hit out of 16 labeled items) and cost 29x more time. For someone who’s going to process hundreds of thousands of files, this trade doesn’t close. 29x slower means a 6-hour job turns into a week-long job.

Here I’ll make a comment about tooling: I did the first round with Claude Code and had asked it to pick the models it thought were best. But then I decided to go to GPT Codex and it made other suggestions I found interesting to test. In summary: I’ve been finding Codex much better for experimentation and exploratory code, for actual research. I find Claude better when we already know exactly what we want.

With Codex, I tested the new candidates qwen3-vl:8b and qwen3-vl:30b-a3b with 3 repetitions for statistical significance. The result? Both worse than qwen2.5vl:7b — type_accuracy of 0.55 versus 0.89 for the incumbent, with a 95% confidence interval that doesn’t even come close. And even slower: 2x and 2.2x respectively. A newer model isn’t always a better model for your use case. qwen3-vl frequently returned truncated or malformed JSON — a real regression in robustness.

OCR: Surya vs Ollama Vision

For text extraction (scanned documents, receipts, screenshots with text), I tested two engines:

Surya finds text in many more files (83% vs 58%), but is 5x slower. When it finds text, the quality is practically the same (similarity > 0.94 on both). Obvious solution: hybrid approach. Use Surya when you need maximum coverage, Ollama Vision as fast fallback. The pipeline design became: try Surya → if it fails or doesn’t find text → fallback to Ollama Vision. That’s why I said at the beginning that Surya is optional, if you don’t want to install it.

Cost Projection

Phase 6 did something I rarely see in open-source projects: cost projection for real usage. With the timings measured per file type, we extrapolated to NAS scenarios:

Take these numbers with some skepticism because it’s back-of-napkin math. $34 in electricity to classify 500,000 files with a local GPU. Try doing that with the GPT-4 Vision API — at $0.01 per image (conservative), that’s $5,000. The price of my setup (GPU + electricity) pays for itself on the first big use.

The lesson: benchmark, don’t guess. I could have assumed the bigger model would be better, or that the newer one would beat the older. The data showed the opposite. ~2 hours of benchmarks saved me from wrong choices that would cost days of rework (in the old days I would have said “weeks”).

Proof of Concept in Python

With the benchmarks in hand, the next step was to validate the complete pipeline before moving to Rust. Not the benchmark pipeline (which tests each model in isolation), but the real classification pipeline — the exact sequence of calls that the final app will make for each file.

I created a Python prototype — 754 lines in a single file (classification/run_classification.py) — implementing the winning strategy from the benchmarks.

The pipeline has 4 enrichment stages:

1. Primary classification: sends the image to qwen2.5vl:7b with a structured prompt, asks for JSON with type, description, tags, confidence
2. Anime enrichment: if the primary type is anime/manga/cartoon, does a second pass with a specialized prompt asking for series, character, scene, artist
3. Document/OCR: if it’s a document or receipt, extracts text with Surya and/or Ollama Vision, then asks for structured data (dates, values, transaction IDs)
4. Output: writes the result in YAML mirroring the directory structure of the source

The most critical part is parsing the LLM’s JSON. Anyone who has worked with LLM output knows they’re… creative with formatting. Sometimes it comes with markdown fences (json ... ). Sometimes there’s text before and after the JSON. Sometimes the JSON is almost right but missing a closing brace. The prototype implemented a 3-attempt cascade that later became the rule for the whole project:

1. Direct parse: json.loads(response) — works in ~70% of cases
2. Brace-balancing extraction: finds the first {, counts opened and closed braces, extracts the substring — catches another ~20%
3. Regex field salvage: if all else fails, uses regex to extract individual fields (“type”: “…”, “description”: “…”) — saves the last ~10%

This cascade stayed in the Rust code practically identical. A well-done PoC shortens the path to production.

Result: 60 images processed, zero errors, 6.29s/image average. The pipeline worked end to end. Time to build the real app.

Building the Tauri App

At this point, I tried to continue with Codex, but then it choked on trying to convert the Python PoC to Tauri. But since the choices had already been made, I went back to Claude Code, and it had no problems mapping from Python to Rust.

Here’s where vibe coding shows what it can do. I’ll tell the real timeline, commit by commit, so you get a sense of the rhythm. The timestamps are from git log, so they’re accurate.

Saturday 02/21 (19:29 → 21:08) — 6 commits, 1h39

It all started with research. Six commits of setup, benchmark scripts, results analysis:

f57221c 19:29 Phase 0: Project setup with uv, environment verification
4977497 19:45 Implement all phases: metadata, image/audio/video classification
af6e3aa 19:49 Add research results report
41d6c2a 20:12 Add per-file timing, OCR phase, cost estimation
0cd1b10 21:02 Fix Surya OCR, re-run on 94-file corpus
25b3ace 21:08 Add conclusions, cost analysis, recommended pipeline

Saturday night, pure research only. Not a single line of app code. But when I went to bed, I knew which model to use, which OCR approach, and how much it would cost in time and electricity.

Sunday 02/22 (13:05 → 23:30) — 14 commits, ~10h25

Sunday was the day of heavy construction. I woke up, had lunch, and sat down to work.

13:05 — First commit of the day: the Python classification prototype. 749 lines validating the complete classification pipeline, the JSON parsing cascade, and the conditional enrichment. This prototype wasn’t throwaway — it was the executable design document for the Rust pipeline that would come later.

I took a break to go out, take a walk, have a glass of wine, then came back and continued:

17:31 — Benchmarks updated with the new candidates (qwen3-vl:8b and qwen3-vl:30b-a3b). Three repetitions each, confidence intervals. They confirmed that qwen2.5vl:7b was the right choice — not by a little, but by a huge margin.

17:56 — The “big bang”: Tauri app scaffold. 9,631 lines inserted in a single commit. The entire app structure: Rust backend with SQLite + FTS5, React frontend, config system, file scanner, data models, query parser for natural search. At that point the app was already searching the database. It didn’t have classification yet, but the foundation was solid — and that’s exactly the point. The scaffold came with tests, TypeScript types, and the directory architecture I defined. Claude generated the code, but the module structure (config, db, scan, models, query_parser) came from how I wanted to organize responsibilities.

19:04 — The heaviest commit of the project: 4,186 lines. Classification pipeline in Rust (1,069 lines of classify.rs — practically the translation of the Python PoC), thumbnail generation (Lanczos3, 300px, JPEG 80%), incremental scanning with fingerprint, the Surya OCR Python script, runtime detection for Ollama, and the brutal expansion of the database with upsert, touch, delete, and FTS indexing. In one commit. With 47 tests. I had to make the architecture decisions (how the cache mirrors the rel_path, how the scan is divided into two phases, how errors propagate), but Claude wrote most of the code and the tests that came with it.

19:55 — UI redesign, scan cancellation, auto-cleanup of orphan classifications, reorganization of the whole repo (moved everything from research to _research_ab_test/). CI and release workflow already configured in this commit — I knew I was going to need them the next day.

21:02 — DB resilience (WAL mode, backup, health check), root management (add, remove, list monitored directories), zoom, sidebar redesign with tree view, thumbnail fix. Four features in one commit. In a manual workflow, each one of these would be a separate PR with days of review.

21:14 → 21:39 — Read-only database mode for sandbox filesystems, resume of interrupted scans on startup, grid tiles redesign with hover overlay, selection model, infinite scroll. Three commits in 25 minutes. Claude was on an absurd pace.

22:41 — Multi-select with Ctrl/Shift click, collage preview with selected items, Ctrl+C copies to the OS clipboard.

23:30 — The big refactor: monolithic frontend (everything in App.tsx - if you don’t explain, Claude always does this) broken into 15 components + 10 hooks + 84 frontend tests. This is the kind of thing that normally takes a full day of tedious work. With vibe coding, it was a one-hour commit. Claude extracted each component, created the hooks, set up the tests with proper mocks, and kept everything working. I just had to say “refactor this monolith into components and hooks, and write tests for each one”.

Monday 02/23 (00:09 → 14:33) — 30 commits, ~14h

Monday was polish, robustness, and the cross-platform marathon.

00:09 — Multi-OS abstraction: all platform-specific code isolated in the platform/ module. This decision, taken early, saved hours of pain in CI. When Windows needed special treatment for UNC paths, the change stayed contained in platform/process.rs instead of spread across 8 files.

00:12 → 01:18 — Quick sequence: cargo audit in CI, duplicate removal, GPL-3.0 license, help dialog (F1) with query syntax examples, sort controls for results, formal SQLite migration system, context menu (copy, delete, rename). Six commits in just over an hour.

09:38 — After sleeping about 8 hours, the first commit of the morning: extraction of the LLM module. The monolith of Ollama calls that lived inside classify.rs was separated into llm/client.rs (HTTP calls, JSON parsing), llm/management.rs (download, listing, cleanup of models), and llm/model_selection.rs (hardware-aware selection by tier). Atomic file ops so cache doesn’t get corrupted if the process dies in the middle of a write.

10:54 — EXIF location extraction (GPS coordinates → readable address), metadata editing modal, setup instructions per OS (each OS has different dependencies for Ollama and Python).

11:34 — PDF support: scanning, indexing, and preview using PDFium. Not trivial — PDFium needs a native binary per platform, page rendering to image, blank page detection (so you don’t generate a thumbnail of an empty cover), thumbnail assembly with the first 2 pages that have content, and native text extraction as a faster alternative to OCR.

12:08 — Albums and Smart Folders. Albums are manual collections (the user drags files in), Smart Folders are saved queries that appear in the sidebar and update automatically. Two database migrations, new sidebar component, drag-and-drop. In 34 minutes.

12:11 → 12:52 — macOS-inspired SVG icons in the sidebar, CLI argument support (sherlock /path/to/folder), copy description/OCR text in the context menu, PDFium path fix in production, icons and screenshots in the README, tauri script in npm, sidebar toggle, dynamic titlebar, Windows compilation fix (Unix-only imports). Ten commits in 41 minutes. Most of these were issues that showed up in CI or during manual testing.

13:14 → 13:54 — The CI fixes marathon. Auto-provision of the Surya OCR venv with progress bar in the SetupModal, icon regeneration with alpha channel, cargo fmt + clippy + UNC paths fix on Windows, tests for help dialog examples, individual folder rescan, Windows assertions fix, release workflow permissions fix. Seven hardening commits. Each one resolving a real bug that appeared in the CI matrix or in testing.

14:26 — Drag-and-drop to reorder roots in the sidebar, scan cancellation before deleting a root (so you don’t leave the scan running in the background on a directory the user removed — a subtle edge case that could cause a crash).

14:33 — Last commit: responsiveness fix in scan cancellation. Check the cancel flag after each classification, immediate poll instead of waiting for the next tick. Small detail, big impact on UX.

Architecture Decisions

Here’s what separates “letting the LLM write code” from “building real software”. None of these decisions came from a prompt. Claude didn’t suggest any of them spontaneously. I had to ask for each one.

Read-Only Principle

The app never writes to the scanned directories. Everything — thumbnails, classification cache, database — sits in ~/.local/share/frank_sherlock/. This is more than good practice, it’s respect for the user’s data. If someone points at the company NAS, the app can’t go around creating .sherlock/ in every subdirectory. If the directory is mounted as read-only via NFS, the app needs to work normally. It sounds obvious, but many cataloging apps you know create caches and thumbnails inside the source directories. (cough Synology @eaDir cough)

Incremental Scanning

Scanning terabytes of data every time the app opens would be insane. The scan is incremental in two senses:

1. Discovery phase (fast): walks the filesystem comparing mtime + size of each file. If nothing changed since the last scan, it doesn’t even read the content — just updates the “seen in this scan” marker. For a NAS with 500K files where 99% hasn’t changed, this phase takes seconds, not hours.
2. Processing phase (heavy): only for new or modified files. Calculates fingerprint (SHA-256 of the first 64KB), generates thumbnail, classifies with the LLM. And here’s where move detection comes in: if a file changed path but the fingerprint is the same, the app preserves the entire classification already done and just updates the path. You reorganized 10,000 photos into new folders? The app detects and doesn’t reclassify any of them.

The checkpoint is per file. If the scan is interrupted (the app crashed, the user closed it, the power went out), the next time it resumes from the last processed file, not from zero. This is implemented via scan job persistence in the database: the scan cursor is saved in scan_jobs.

Cooperative Cancellation

The scan runs on a separate thread via tokio::spawn_blocking. To cancel, I use an AtomicBool shared between the scan thread and the frontend. The flag is checked:

• Before each file in the discovery phase
• Before each classification in the processing phase
• After each classification (in case the Ollama call takes a while)

This ensures cancellation responds in at most the time of one classification (~1 second), not the time of the whole scan. Without this design, canceling a scan of 500K files could take minutes — or simply not work.

Database Resilience

SQLite with WAL mode (allows concurrent reads during writes), health check on startup, automatic backup before migrations, formal migration system via rusqlite_migration. Five migrations in total:

0. Initial schema (files, roots, scan_jobs tables, and the virtual FTS5 table for search)
1. location_text column for EXIF address
2. FTS index rebuild (needed after changing tokenization)
3. albums and album_files tables for manual collections
4. smart_folders table for saved queries

Migrations are identified by position and can never be edited or reordered after being published. This is the kind of rule you learn after corrupting someone’s production database once. The rule is coded in the project’s CLAUDE.md so future vibe coding sessions don’t violate it.

Hardware-Aware Model Selection

Not everyone has an RTX 5090. The app detects the GPU on startup and picks the appropriate model:

• Weak GPU or no GPU: qwen2.5vl:3b (small tier — runs on anything)
• GPU with >= 6GB VRAM: qwen2.5vl:7b (medium tier, the benchmark default)
• Apple Silicon with >= 48GB unified: qwen2.5vl:32b (large tier, only where unified memory allows without swap)

Detection uses nvidia-smi on Linux/Windows, system_profiler on macOS, and sysinfo as fallback for system RAM. The result is cached in the Tauri AppState so it doesn’t keep running subprocesses all the time.

Multi-OS and CI/CD

If there’s one part of the project that justifies having experience, it’s this one. Making software that compiles is easy. Making software that compiles and passes all tests on Linux, macOS, and Windows at the same time teaches you humility.

Platform Module

All OS-specific code lives in src-tauri/src/platform/:

• gpu.rs: GPU detection (NVIDIA via nvidia-smi, AMD via sysfs/rocm-smi, Apple via system_profiler)
• clipboard.rs: image copy to clipboard (xclip on Linux, pbcopy on macOS, PowerShell on Windows)
• python.rs: Python location (python3 vs python in PATH), venv paths per OS
• process.rs: subprocess execution abstraction with output handling

This means classify.rs, scan.rs, thumbnail.rs — none of them know which OS they’re running on. They ask the platform and the platform resolves it. When Windows needed special treatment for UNC paths (those that start with \\?\), the change stayed contained in platform/. When macOS needed a conditional import, same thing. The rest of the codebase wasn’t touched.

GitHub Actions Matrix

Two workflows:

• CI (push + PR): build and test on Linux, macOS, and Windows. Each push runs cargo test + npm test on the 3 OSes. Includes cargo fmt --check, cargo clippy -- -D warnings, and cargo audit. If any platform fails, the PR doesn’t pass.
• Release (tags v*): build via tauri-action, generates AppImage (Linux), DMG (macOS arm64), MSI (Windows), and creates a Draft Release on GitHub with the binaries attached.

The 10+ CI fix commits on Monday morning were the least glamorous part of the project. Things like:

• #[cfg(not(target_os = "windows"))] on imports that use std::os::unix::fs::PermissionsExt
• dunce::canonicalize instead of std::fs::canonicalize because Windows generates paths with \\?\ prefix that break string comparisons
• Install rustfmt and clippy explicitly on the runner because they don’t always come in the default GitHub Actions toolchain
• Remove the macOS Intel target from the release workflow (Apple Silicon only — not worth the cost of maintaining two Mac targets)

Nobody posts these things on X. But without them, your app doesn’t build on 2 of the 3 targets.

External Integrations

The app depends on 3 external systems. Each one brought its own problems.

Ollama

Ollama serves the vision models via local REST API (port 11434). The app does:

• Status check: checks if Ollama is running and lists installed/loaded models
• Model download: if the recommended model isn’t installed, offers download with a progress bar via API streaming
• Generation: sends image in base64 + prompt, receives JSON (with the 3-level parsing cascade)
• Cleanup: unloads models from VRAM when not classifying, so it doesn’t monopolize the GPU from the user’s other applications

Ollama is the only hard requirement. Without it, classification doesn’t work. The SetupModal guides the user through installation and model download.

Surya OCR

Surya is a Python OCR engine that runs locally. The problem: the app is Rust and can’t depend on a system Python installation. The solution:

• The app maintains an isolated Python venv at ~/.local/share/frank_sherlock/surya_venv/
• The surya_ocr.py script is bundled as a Tauri resource (packaged in the binary)
• On first use, the SetupModal offers to automatically provision the venv (finds Python, creates venv, pip install surya-ocr + dependencies)
• Classification calls the script via subprocess, passes the image as argument, reads the extracted text from stdout

Surya is a soft requirement: if it’s not installed, the app works normally — it just won’t have dedicated OCR. The pipeline gracefully degrades to use Ollama Vision as fallback, which is worse in coverage but works. The user sees a warning in setup, not an error that blocks usage.

PDFium

For PDFs, I needed native text extraction and page rendering for thumbnails. PDFium is Chrome’s PDF engine, and has Rust bindings via pdfium-render.

The PDFium binary is downloaded by a script (scripts/download-pdfium.sh) and bundled via lib/ in Tauri resources. Each platform gets the correct binary (.so, .dylib, .dll). lib/ is gitignored — the binaries are downloaded at build time, not versioned.

The PDF pipeline:

1. Tries to extract native text (no OCR) — many PDFs already have a text layer
2. If there’s enough text, uses it directly for indexing (faster and more accurate than OCR)
3. If not, renders the page and sends it to the image pipeline (Ollama Vision)
4. Detects blank pages, finds the first page with real content
5. Generates thumbnail as a montage of the first 2 pages with content (gives a better sense of the document than just the cover)

What Agile Vibe Coding Really Is

OK, now the point that really matters. The reason I’m writing this.

What Claude Did

• Wrote most of the Rust and TypeScript code
• Generated 166 Rust tests and 172 frontend tests
• Implemented JSON parsing with 3 fallback levels
• Set up CI/CD with 3-OS matrix
• Did massive refactors (monolith → 15 components + 10 hooks)
• Handled edge cases of encoding, Unicode paths, and exotic file formats
• Wrote SQL queries, migrations, FTS5 indexes
• Implemented GPU detection, clipboard per OS, Python resolution
• Created the setup flow with progress bar and model download
• Debugged and fixed dozens of cross-platform compilation issues

The speed is hard to describe without sounding like exaggeration. Saturday’s 19:04 commit, the one with 4,186 lines and 47 tests, took about an hour including my review. A human dev, even a good one, would take a full day to write that with the same test coverage.

What I Did

• Decided to do benchmarks before writing any app code
• Chose Tauri over Electron (smaller footprint, native Rust, no Node runtime in prod)
• Defined the read-only principle as an inviolable rule
• Designed the incremental scan with move detection via fingerprint
• Decided on cooperative cancellation with AtomicBool
• Demanded formal schema migration (no ad-hoc ALTER TABLE in loose scripts)
• Insisted on platform abstraction from the first multi-OS commit

And mainly: I asked the annoying questions. “What if the scan is canceled in the middle?” became the checkpoint system. “What if the database corrupts?” became WAL + backup + health check. “What if the person doesn’t have a good GPU?” became tier model selection. “What if Surya isn’t installed?” became a soft requirement with fallback. “What if the user deletes a root that’s in the middle of a scan?” became cancel-before-delete. “What if the file moves but the content is the same?” became move detection.

Oh, and I decided when to stop adding features and publish.

That last one is underrated. The temptation to keep adding “just one more thing” is enormous when the marginal cost of implementing is low. Claude implements any feature I ask for in minutes. But software that’s never published is useless to anyone. Knowing when to stop is a skill no LLM will give you.

The Real Pattern

Agile Vibe coding isn’t “asking the LLM to make an app”. It’s pair programming with a partner who’s stupid fast, has perfect memory, and never complains about refactoring. You say what you want, it implements, you review and adjust the direction. But if you don’t know where to go, speed doesn’t help — you just get to the wrong place faster.

The questions I asked — about resilience, cancellation, multi-OS, graceful degradation, edge case detection — none of them came from the LLM. They came from years building software that has to work in real environments, with real users, on diverse hardware. Claude isn’t going to ask you “what if the user pulls the network cable in the middle of downloading the model?”. But if you ask, it implements the handling in minutes.

It’s tempting to look at this project and conclude that anyone with a good idea could have done the same. But try to imagine: without the decision to do benchmarks, I would have picked the wrong model. Without the Python PoC, I’d have discovered JSON parsing problems in production. Without the platform abstraction, I’d be debugging Windows issues scattered across 15 files. Without the scan checkpoint, users would lose hours of processing on every crash. Without the formal schema migration, the first update would break everyone’s database.

Think of an architect with an absurdly efficient construction crew. The architect doesn’t need to lift every wall, but needs to know where they go and what happens if you take one out. The crew executes fast, works at night, doesn’t complain. But someone needed to have drawn the blueprint. Without a blueprint, it’s just a pile of bricks stacked quickly.

Final Numbers

For those who like concrete numbers:

The first commit was Friday 02/21 at 19:29. The last was Monday 02/23 at 14:33. Discounting sleep (~8h on Saturday/Sunday night, ~8h on Sunday/Monday early morning) and breaks, that’s ~26 hours of work distributed over a weekend.

From zero — without a single file in the repository — to published binaries for 3 operating systems, with automated tests running in CI on every push. Including the research phase, which on its own would justify a sprint.

Conclusion

Agile Vibe coding works. But it works like any powerful tool: in the hands of someone who knows what they’re doing.

The idea for Frank Sherlock would fit in a tweet: “classify images with local LLM”. But turning that into real software required: benchmarked research with formal ground truth, proof of concept validating the complete pipeline, incremental architecture, error handling at 3 levels, cooperative cancellation, formal schema migration, platform abstraction, CI/CD with 3-OS matrix, integration with 3 external systems with graceful degradation, and 338 tests to ensure none of that breaks when someone runs cargo update.

The LLM sped all that up absurdly. But it didn’t replace the need to know what to do. If I had asked “make an app that classifies images” without the 2 hours of benchmarks, without the proof of concept, without the architecture decisions, without the annoying questions about edge cases, the result would be a prototype that works on my machine and breaks anywhere else. And I probably wouldn’t even notice until someone complained.

The vibe needs an experienced conductor. For now, that conductor is still human.

Code at github.com/akitaonrails/FrankSherlock. GPL-3.0, local-only, open source.

And make sure you subscribe to my new newsletter The M.Akita Chronicles!

–

274 commits. 8 days. 4 applications. 1,323 tests. A complete system in production — with real subscribers receiving a newsletter every Monday at 7am, an AI-generated podcast on Spotify, a static blog on Netlify, and a Discord bot that’s my operational interface.

I did this with a partner: Claude Code, an AI agent running in the terminal. And the most important lesson I took away is not that “AI programs by itself” — AI programming by itself is a recipe for disaster. What works is something that already has a name and decades of history: Extreme Programming. Except now the pair in pair programming isn’t human.

I need all of you to go through an experience: a 1-WEEK IMMERSION completely DETACHED from the code, using AI as pair programming, going from zero to production on any small project. Trust me, until you do it at least once, you won’t get it. But after that, everything changes.

The “One-Shot Prompt” Myth

There’s a growing narrative that “vibe coding” means writing a detailed spec, handing it to the AI agent, and it delivers the system ready. Like those “I built a SaaS in 10 minutes with Cursor” videos. It looks magical until you look at what was actually delivered: a prototype with no tests, no security, no error handling, no deploy, nothing that makes a system survive contact with real users.

The reality of my 274 commits tells a different story:

Only 37% of the commits are features. The rest is the work that makes real software: fixing bugs, hardening against failures, protecting against attacks, deploying, testing, documenting. Anyone showing only the features part is selling an illusion.

More than that: many of the features emerged because iteration revealed they were needed. ContentPreflight wasn’t in the original plan — it was born when the second newsletter went out with an empty section because a job failed silently. RecoverStaleDeliveriesJob was born when I realized a crash in the middle of sending left emails in eternal limbo. StealthBrowser was born when Morningstar blocked HTTParty. No spec in the world predicts this. The correct system emerges from iteration, not from specification.

Pair Programming With an AI Agent

I don’t use Claude Code as a code generator. I use it as a pair programming partner.

The difference is fundamental. A code generator takes a spec and returns output. A pair programming partner talks. I interrupt when it takes the wrong path. I give it context it doesn’t have. I question decisions. I ask for alternatives. Sometimes I say “stop, this is getting too complicated” and it simplifies.

Some real examples from the 274 commits:

Interruption for over-engineering. When I asked for the email delivery system, the first proposal had a state machine with 8 states, separate retry queues, and dead letter handling. I interrupted: “Simplify. Four states: pending, sending, sent, unknown. When in doubt, don’t resend.” The result was SendSingleEmailJob with the ses_accepted variable as the pivot — 40 lines that solve the whole problem.

Context the agent doesn’t have. When we were on Yahoo Finance, Claude tried to use HTTParty. I explained: “Yahoo does TLS fingerprinting, it’ll block any HTTP client that isn’t a real browser.” Result: headless Chromium with crumb authentication. That’s in no tutorial — it’s the kind of knowledge that comes from trying and failing.

Joint architecture decision. The podcast pipeline could have been a Python monolith. I suggested: “Rails orchestrates, Python serves TTS over HTTP, content integrates by filesystem.” The result — QwenTtsClient in Ruby calling podcast-tts in Python — keeps each language doing what it does best.

Iterative prompt refinement. The first newsletters came out generic. The Akita and Marvin comments sounded the same. I tuned prompts with it: “Akita never says ‘maybe’. Marvin uses ‘well…’ and ‘anyway’. The LLM will soften opinions if you’re not explicit.” Each tweak was a prompt commit, tested with real data.

The pattern is: I bring the what and the why. The agent brings the how — and the how is often better than I’d do alone. But how without what produces correct code that solves the wrong problem. And what without how produces specs that never leave paper.

Extreme Programming Isn’t a 2000s Fad

Everything we did has a name in XP. Not because I planned it — because these are the practices that naturally emerge when you program with fast feedback:

Pair Programming

Already explained. The pair is the agent. The dynamic is the same: one drives, the other navigates. I navigate more than drive — I define direction, question decisions, correct course. The agent drives more than navigates — writes code, runs tests, proposes solutions. When it inverts (me dictating exact code, it just typing), the result always gets worse.

Small Releases

Out of 274 commits, every single one passes CI. No exceptions. There’s no “broken commit that’ll be fixed in the next one”. Every commit on master is production-ready.

CI runs on every commit: RuboCop (style) → bundler-audit (gem vulnerabilities) → Brakeman (static security) → full tests. It’s ~22 seconds. Sounds like nothing, but multiplied by 274 commits that’s 100+ minutes of automated validation I didn’t have to do manually.

The distribution of commits by day shows the rhythm:

The day-2 “explosion” isn’t chaos — it’s small releases stacked up. Each one adds a feature, runs CI, is ready to deploy. It isn’t a 2-week waterfall followed by a big-bang merge. It’s real continuous integration.

Test-Driven Development

1,323 tests. 970 in marvin-bot (6 seconds in parallel), 353 in newsletter (1 second in parallel). Ratio:

More lines of test than of code. In both applications. And this isn’t vanity — it’s the infrastructure that enables speed.

Coverage from SimpleCov:

It’s not 100% — and never will be. The 17-18% uncovered is mostly integration with external APIs (Discord, SES, OpenRouter) that are mocked in tests. Real coverage on business logic is above 95%.

The question that matters: are these tests useful? The empirical proof: over 274 commits, I saw CI fail and catch a real bug more than 50 times. Without those tests, every one of those bugs would have gone to production. The agent proposes a change, the test catches the regression, the change is fixed before committing. Without tests, the agent hands you plausible code that silently breaks something that worked yesterday.

TDD with an AI agent is multiplicative. The agent is good at generating tests — it knows the patterns, knows the common edge cases, knows the Minitest structure. And the tests it generates become the safety net for the changes it makes later. It’s a virtuous cycle: tests enable speed, speed generates more tests.

Continuous Refactoring

27 explicit refactoring commits. Some examples:

• Extract StealthBrowser wrapper — The headless browser code with stealth patches was duplicated in 4 clients. Extracted into a service, the 4 clients were left with 5 lines each.
• DRY section rendering: collapse 5 bubble methods + 7 Hugo shortcodes — 5 bubble rendering methods in MarkdownToHtml that were copy-paste with different colors, collapsed into 1 parameterized method. 7 Hugo shortcodes delegating to 1 partial.
• Replace market recap LLM generation with Yahoo Finance real data — The LLM invented market numbers. Replaced with real data from Yahoo Finance + LLM commentary. Better result, lower cost.
• Replace DuckDuckGo with Brave Search API — DuckDuckGo doesn’t have a stable API. Brave Search has an official API with 2,000 queries/month free. Clean swap.
• Rework ArticleFetcher: browser-first with anti-detection — Two full rewrites. The first tried HTTP-first with fallback to browser. The second inverted: browser-first because most reader links are SPAs. Each rewrite kept tests passing.

Refactoring is where the pair programming partner shines. I say “this code is duplicated, extract a concern” and the agent does in 2 minutes what would take me 20. But I decide what to extract and how the interface should look. The agent alone doesn’t refactor — it piles new code on top of existing code.

Continuous Integration

The CI already described above: rubocop + bundler-audit + brakeman + tests. Runs on every commit, in both apps. Brakeman deserves a mention:

Zero security warnings in both. The 2 ignored in newsletter are documented false positives (SQL interpolation with config paths, not user input).

Brakeman caught real problems during development: SQL injection in a search query, path traversal in the images controller, open redirect in unsubscribe. Each one fixed before going to production, in the same commit where CI flagged it.

The Anatomy of a Feature: From Request to Deploy

Let me take a concrete example to illustrate how the flow works in practice. List-Unsubscribe (commit 5950ccc).

Context: confirmation emails weren’t reaching Gmail. Everything green — DKIM, SPF, DMARC, SES accepting. But Gmail was silently dropping them into spam.

Investigation (me + agent): we researched. Since February 2024, Gmail requires List-Unsubscribe + List-Unsubscribe-Post on every bulk sender email. Without them, it penalizes the entire domain — including transactional emails.

Implementation (agent executes, I validate): header in SesMailer, RFC 8058 endpoint in UnsubscriptionsController (skip CSRF for email clients, returns 200 instead of redirect), tests covering both flows.

CI: rubocop OK, brakeman flagged the skip_forgery_protection as a warning (correct — we validated it’s conditional). Tests pass.

Deploy: kamal deploy. In 2 minutes, the fix is in production. Domain reputation takes 1-2 days to recover. The next day, confirmation emails start arriving again.

Total time: ~25 minutes from “why is Gmail rejecting this?” to production deploy. Without the agent, it would be hours of RFC reading, manual implementation, and hoping I didn’t forget anything.

What AI Does Well (And What It Does Badly)

After 274 commits with an AI agent, I have a pretty clear sense of the strengths and weaknesses:

Does well:

• Boilerplate and scaffolding: jobs, services, tests, migrations — the agent produces at typing speed
• Tests: surprisingly good at identifying edge cases and generating comprehensive tests
• Mechanical refactoring: renaming, extracting methods, moving code between files — fast and precise
• Contextual research: “how does RFC 8058 work?” produces an actionable answer in seconds
• Consistency: follows patterns established in the project (concerns, naming conventions, test structure) without forgetting

Does badly:

• Architecture decisions: tends to over-engineer. Needs constant human braking
• Specific domain knowledge: doesn’t know that Yahoo does TLS fingerprinting or that Qwen’s VoiceDesign produces a European accent
• Opinions: softens everything. Personality prompts become generic mush without explicit instructions
• Proactive security: implements what you ask for, but rarely suggests protections you didn’t ask for (SSRF, rate limiting, encryption at rest)
• Prioritization: executes anything with equal enthusiasm. Never says “this is a waste of time” or “do X before Y”

AI doesn’t program by itself, and it isn’t useless. It’s a programmer multiplier, not a substitute. A mediocre programmer with AI produces mediocre code faster. An experienced programmer with AI produces good code at a speed that was previously only possible with a team.

The Numbers That Matter

Let’s look at the real size of what was built in 8 days:

Code

Total functional code: ~16,000 lines. Total with tests: ~36,000 lines.

Components

• 43 services in marvin-bot (scrapers, AI, parsers, validators, uploaders)
• 38 jobs in marvin-bot (content generation, daily digest, podcast, infrastructure)
• 12 services in newsletter (email delivery, publishing, assembly)
• 11 jobs in newsletter (sending, recovery, backup, blog publishing)
• 23 AI prompt templates (personality, generation, podcast, tools)
• 17 Hugo shortcodes (section rendering on the blog)
• 3 RubyLLM tools (image generation, web search, web fetch)

Pipeline

• 10 content sections generated per week (anime, hacker news, youtube, market, world events, history x3, holidays, closing remarks)
• 10 daily digest jobs posting to Discord at 8am
• 1 podcast of ~30 minutes generated per week (script + TTS + assembly + S3)
• 1 newsletter assembled, published to the blog, and sent by email every Monday

Is This 200 User Stories?

I looked at the 274 commits and tried to map them to user stories in the agile sense. Not every commit is a story — many are fixes inside the same feature, prompt tweaks, or CI cleanup. But the conservative count of distinct features, each one with implicit acceptance criteria (works, has tests, passes CI, deploys):

Estimate: ~180-200 user stories.

That includes:

• Each newsletter section (10 distinct generators with scraping, AI, validation, fallback)
• The email pipeline (delivery tracking, atomic claiming, recovery, List-Unsubscribe)
• The podcast pipeline (LLM script, dual-model TTS, assembly, S3, RSS)
• The Discord bot (15+ commands, embeds, context, multi-turn /ask, daily digest)
• The blog (Hugo + Hextra, 17 shortcodes, RSS podcast, SEO, GA)
• The subscription page (confirmation, themes, LGPD, terms of service)
• Infrastructure (Kamal deploy, Cloudflare, SES config, RunPod, backups)
• 21 security commits (SSRF, CSP, rate limiting, encryption, path traversal)

In traditional Scrum with a team of 2-3 devs, a well-done story (with code review, QA, deploy) takes on average 1-2 days. 200 stories × 1.5 days / 2 devs = ~150 days of work, or ~30 weeks in 2-week sprints (counting planning, review, retro, and the inevitable overhead of people coordinating with each other).

Being generous — senior team, well-defined stories, no impediments — maybe 10-15 weeks.

We did it in 8 days. One person and one AI agent.

But — and this is crucial — I’m not just any dev and this isn’t just any project. I have 30 years of software experience. I know which decisions matter and which don’t. I know when to stop engineering and ship. I know when the agent is wrong before it finishes generating the code. And the project is solo-developer, without external code review, without separate QA, without coordination between teams.

The agent’s real multiplier isn’t “10x more code”. It’s eliminating dead time. Without an agent, 70% of my time would be typing boilerplate, reading API docs, writing mechanical tests, debugging typos. With an agent, that time goes to zero and I focus 100% on architecture, domain, and quality decisions.

The CLAUDE.md: The Spec That Evolves

One thing that works exceptionally well with AI agents is having a living document that describes the project. The project’s CLAUDE.md has 702 lines and covers:

• Architecture overview
• Full tech stack
• All environment variables
• Content directory structure
• Services, jobs, and models of each app
• 12 “common hurdles” with documented solutions
• 14 project design patterns
• Full weekly pipeline with schedules
• Post-implementation checklist

This document wasn’t written all at once. It evolved with the project. Every time we found a hurdle (Yahoo Finance needs Chromium, HN RSS changes format, LLM softens opinions), we documented it in CLAUDE.md. In the next session, the agent already knows.

It’s the equivalent of a team’s onboarding doc — except the “new team member” is the agent, and it reads the whole doc in 2 seconds before every interaction. Investing in documentation pays back exponentially with AI agents because they actually read it.

The Lessons

1. Vibe coding without discipline is a throwaway prototype. 274 commits with CI, tests, security scanning, and small releases is the opposite of vibe coding. It’s software engineering with a copilot that types fast.

2. TDD is more important with AI, not less. The agent modifies code with confidence because it has 1,323 tests as a safety net. Without tests, every change is a gamble.

3. The human decides the what. The agent decides the how. Flip it and the result drops dramatically.

4. Continuous refactoring is mandatory. The agent piles code. If you don’t prune regularly — extract concerns, DRY duplications, simplify interfaces — the codebase becomes a monolith of 500 lines per file.

5. Documentation is an investment with immediate return. Every hour documenting in CLAUDE.md saves hours of lost context in future sessions.

6. Small releases are the key. Every commit is production-ready. If something goes wrong, revert one commit. If the feature was a bad idea, it never depended on another. This only works with automated CI.

7. Security isn’t a phase — it’s a habit. 21 security commits spread across 8 days, not concentrated in a “security sprint” at the end. Brakeman runs on every commit. Vulnerabilities are fixed the moment they appear.

8. The agent never says “no”. That’s a bug, not a feature. If you ask for something over-engineered, it implements with enthusiasm. If you ask for something insecure, it implements without complaining. You are the brake. You are the code review. You are the adult in the room.

The Counter-Example: FrankMD

Everything I described above may sound theoretical. “Of course TDD and continuous refactoring are important” — everyone agrees in theory. So let me prove it empirically with a counter-example from the same developer, with the same AI agent, two weeks earlier.

Before the M.Akita Chronicles, I built FrankMD — a Markdown editor in Rails, Obsidian/VSCode style. 212 commits in 19 days. Same setup: me + Claude Code as a pair programming partner.

The difference: on FrankMD, I did what any developer does when “it’s just a little personal project”. I focused on features. Tests? Later. Refactoring? When necessary. CI? Runs when I remember.

The app_controller.js Spiral

FrankMD is a SPA-like with Stimulus. In the beginning, all the JavaScript lived in a single file: app_controller.js. Here’s what happened to it across commits:

The pattern is clear: build-build-build-STOP-refactor. The file grew 10x in 54 commits, hit almost 5,000 lines, and then I needed 6 rounds of refactoring to bring it back to acceptable. Each stop was a big commit (hundreds of lines moved, dozens of files created), not a surgical tweak.

The CSS followed the same pattern: 1,053 lines in a single application.css, until the refactoring commit that exploded it into 10 theme files + 9 component files.

Tests: Before vs. After

On FrankMD, the 2nd commit was “Add comprehensive test suite” — system tests (browser integration). Sounds responsible, but those tests covered the happy path and nothing else. The JavaScript unit tests only came in commit #52, after app_controller.js had already hit 5,000 lines — because I needed tests to refactor safely. Retroactive tests.

Ruby coverage reflects this: it had to be pushed from 77% to 89% in an explicit “Add tests for untested success paths” commit. That wasn’t TDD — it was filling holes afterward.

The Numbers Side by Side

The number that matters: commits per day. The M.Akita Chronicles had 3x more throughput — with TDD, CI, and continuous refactoring — because it never had to stop. No “stop everything and refactor 5,000 lines into 10 files” commit. No “I need to add retroactive tests so I can touch this code without fear” moment.

The Concrete Lesson

FrankMD’s 6 big refactors weren’t “continuous refactoring”. They were emergency surgery on accumulated technical debt. Each one took hours — not the 2 minutes of the “extract a concern” described above. And each one introduced risk: when you move 1,500 lines from one file to 10 new files, the chance of breaking something is proportional to the size of the change.

With TDD and continuous refactoring, the M.Akita Chronicles never accumulated that debt. The 27 refactors are small commits — “Extract StealthBrowser”, “DRY bubble methods” — that take minutes and are protected by the 1,323 tests that already existed when the refactoring was done.

Same developer. Same AI agent. Two different approaches. Result: the disciplined project delivered more, in less time, with less pain.

Conclusion

This project isn’t an AI showcase. It’s a showcase of software engineering accelerated by AI. And the strongest proof is that the same developer with the same agent produced radically different results depending on the process.

On FrankMD, without discipline, the result was predictable: a 5,000-line file, 6 emergency surgeries, retroactive tests, and 11 commits per day. On the M.Akita Chronicles, with XP, the result was 34 commits per day, zero forced stops, 1,323 organic tests, and a production system in 8 days. The variable wasn’t the AI — it was the process.

274 commits in 8 days don’t happen because the agent is magical. They happen because Extreme Programming — pair programming, TDD, small releases, continuous refactoring, continuous integration — works as well with an AI pair as it does with a human pair. Maybe better in some ways: the agent never gets tired, never argues from ego, never says “but we’ve always done it this way”, and reads 700 lines of documentation before every interaction without complaining.

The future of development isn’t “AI replacing programmers”. It’s programmers who know how to use AI as a tool — with the same discipline they’ve always used with any other tool — producing in a week what entire teams take quarters to deliver.

Not because the AI is brilliant. Because the process is brilliant. XP was right from the start. It was just missing a pair that never gets tired and types at 1,000 words per minute.

Remember:

“AI is your mirror, it reveals faster who you are. If you’re incompetent, it’ll produce bad things faster. If you’re competent, it’ll produce good things faster.” - by Akita

The TL;DR is this: in the old days, if I were to build this web app from scratch, it would have cost me around 200 hours, spread over 1 or 2 months, going slow, back and forth. And it probably wouldn’t have ended up as complete as this one.

With Claude, I did it all in a little over 30 hours, spread across 3 days of super hyper focus (I don’t recommend it; I almost got sick doing it). In the end, this produced a project with roughly 130 pull requests. A total of 18 thousand lines of code (test coverage around 70%). It was all done in Ruby on Rails 8 and JavaScript (HotWire/Stimulus).

There are several advantages to this particular app: it’s a very well-known and well-documented scope. There are hundreds of text editors of all types and complexities. Surely every LLM has been trained with tons of material on this. So when I ask it to search by filename, it already knows to reach for something like the Levenshtein algorithm for fuzzy matching, for example.

Searching and listing images in a grid, uploading to S3, find and replace, HTML preview, etc. All of this is super well documented and trivial to do. Even without LLMs, any intern would find it in the first Google link and on Stack Overflow. So I’d say more than 90% of the scope was easy. An app with a less well-known scope — say, a scientific application — would have been much harder. So this example of mine is only slightly above a Hello World: repeating a well-known scope that already exists.

Unlike what I did in my first Vibe Coding article, this time I won’t comment commit by commit; there are more than 130! We’d be here all week. Instead, I’ll comment on a few themes that catch my attention and that might be relevant to other projects.

Choose Your LLM Plans Carefully

Over those 3 days, I was interrupted by Claude Code about 2 or 3 times and had to use something else for 2 or 3 hours. That’s because it has some daily token limit and tells me to stop after I use too many tokens.

I’m using the Max Plan (5x Pro), which I think costs around USD 100, meaning around USD 6 a day. In this case it was cheap, because 3 days would only cost USD 18.

But during one of those breaks, I switched to using Claude Opus via OpenRouter, and that was WAY more expensive. Outside the Claude Code exclusive Max plan, the per-token cost is absurdly higher. In about 2 or 3 hours, it already burned through more than USD 100 (equivalent to the whole monthly subscription!)

So, in total, by using the Max plan outside its bounds, I ended up spending more than USD 120. But if I had just stopped to rest between the daily limits, I would have spent one extra day (wouldn’t have gotten sick 😅) and wouldn’t have gone much beyond USD 20. So you need to research the plans for each platform carefully before jumping in without thinking, like I did. 😂

As I’ve said over and over: when in doubt, use Claude Code with the Max plan.

Choose Your Architecture Carefully

When I started coding (every time I say “I coded”, read it as “I vibe coded”, of course), the start was very fast. I decided only a few technical things: I wanted it to be Ruby on Rails 8, I wanted it to use Tailwind CSS, I wanted it to use no database and implement everything based on the filesystem, I wanted it to have no authentication, since it would be self-hosted for my use only.

The first 10 commits took a little over an hour and a half. I kept adding features bit by bit to give “shape” to the editor, and at this point I already had a minimal, good-looking editor, with a dark theme, that edited and displayed an HTML preview. Not bad for a prototype.

As I said in the other article, this would be one of the “it already works” milestones.

But of course, there are still more than 120 commits to really work, and nearly 30 hours to reach the “really works” stage.

I kept adding various small features, in this order:

• Add a Markdown cheat-sheet (help)
• Dialog box to add fenced code blocks with language autocomplete
• Theme system, already adding around 10 themes (but not yet tuned for Omarchy)
• Dialog box to customize fonts
• Zoom control on the Preview
• First attempt at “typewriter mode” (this will cause plenty of headaches later)
• Dialog box to search files by path with a fuzzy finder
• First attempt to sync the editor scroll with the preview (this will also cause plenty of headaches)
• Dialog box to search files by content with regex support
• Dialog box to search and embed YouTube videos
• Dialog box to search images, upload and insert
• Initial Docker support

That completes the first 34 commits and about 4 hours of work. Again, very close to “it already works”.

But the fun begins at commit 7bbf03a, titled: “Refactor to Restful Architecture”.

This is a Rails-specific thing, and an architecture thing in general. Claude, without me instructing it, did exactly what I’d expect a Rails intern to do: it crammed all the logic for managing directories and files, image uploads and everything else into 2 or 3 service modules.

It also wrote all the routes manually. That is, it didn’t use the most basic thing in Rails: the concept of Restful Resources. Read the linked guide; I won’t explain what that is here, just that any experienced Rails person knows it with their eyes closed.

So it created routes like “http://localhost:3000/notes” but didn’t enable things like “http://localhost:3000/notes/hello” automatically opening “hello.md”, and HTTP verbs like PATCH to update the file or DELETE to remove the file. And also, without the GET option, I wouldn’t have the option to bookmark a file directly. All of this comes for free if you use RESTful. So that commit was the refactor where I instructed it to redo everything in that architecture. Just that alone was almost 1000 lines added.

The “correct” thing would have been to set up, right at the start, a CLAUDE.MD or CONTRIBUTING.MD file or something similar with the specific architecture rules and code conventions — but I didn’t, so here’s a reminder for you. 😅

Refactoring and Fixes

After that first big refactor, I went back to making adjustments (fixes) and kept adding new features:

At this point we already crossed 47 commits and more than 8 hours non-stop. And yes, I didn’t take my eyes off the monitor. You can’t read 100% of everything Claude is thinking and doing; it’s a lot, but overall I’ve noticed you need to keep glancing at it and, most importantly, pay very close attention!

The processing time for multiple agents in parallel, many in reasoning/thinking mode, is really slow. Claude is one of the fastest compared to the others, but in practice I find it painfully slow.

After lunch, I decided to open YouTube on the side while waiting for Claude to finish the tasks. That means I wasn’t paying attention and only glanced over to see if it finished with “success” or asked me for some permission to execute something.

I don’t remember if it was at this point, but I went to look at what it had generated, and of course, by this stage it already had an application.css with more than 1000 lines, which I asked it to fix in this commit, and it broke one huge file into all of these:

Even the themes: everything was in a single file!! This is the level I’d call “intern”. Since everything “works”, it just keeps piling up code, all in the same place, and leaves worrying about it for later. Every intern does this at the start if nobody teaches them, and apparently the LLM does the same. It won’t proactively do anything you didn’t explicitly ask for. In fact, I never said in the prompt “remember to keep breaking things into small files”, but I should have.

But that wasn’t the worst. No sir, this commit has all the JavaScript in A SINGLE FILE. I think it was already past 4 or 5 thousand lines. Look at how many files it broke it into, it doesn’t even fit in this picture:

The problem is that I myself fell into what I’m criticizing here. I’d ask for a new feature, hit “reload” on the browser and voilà, there was the new feature! “WORKING”!! (Ruby on Rails is very productive!)

And that’s how it went, adding “it already works”, adding, “it already works”, and when I finally looked, horror show!!

At the last refactoring we were at around 9 hours of work. This is where we wrap up the day; we’re on the next day and, not counting the time I slept, we’re already at 17 hours, 70 commits. Roughly HALFWAY through the project.

All this to say the following: the BIGGEST part of Vibe Coding will always be REFACTORING, fixing bugs, and adding TESTS!

I18n - Internationalization

I got a little ahead of myself; before the JavaScript mega-refactoring, I had done another mega-refactoring: I asked it to extract all the English strings and already translate a Brazilian Portuguese option for me. Another gigantic commit with more than 1600 lines modified.

I wanted to make a separate section just to comment on this because this is one of the points where I think LLMs shine. They were originally built precisely to deal with natural language and translations. LLMs are very good with languages. In the old days I’d have to go through Google Translate, which was always pretty bad, and then find a native speaker to adjust the terms Google got wrong.

As expected, after extracting everything, asking it to adjust the tests to take i18n into account, and seeing that Portuguese was working, I started adding others just for fun and was very pleased with the result. I18n is definitely something that doesn’t make sense to do without an LLM these days. It’s orders of magnitude better than the old way.

The Last HALF

From the last big refactor to the end, there are about another 70 commits and roughly 14 hours, the end of the second day and the third day, with a roughly 15-hour break in the middle where I got a fever and had to throw myself into bed and stuff myself with medicine… but all good, the next morning I felt a bit better and finished it!

From there on, I kept adding the rest of the features, like emoji, emoticons, find and replace dialog, jump to line dialog, adjustments to the configuration system (via envs, via .fed), and Docker.

One thing I didn’t mention before, but that came before the I18n effort, was a REBRANDING. This is another thing LLMs are very good at solving: text substitution, knowing how to tell apart plain text, variables and code in general. I wasn’t very inspired and, at the start, I was going to make a MUCH simpler little web editor than what it ended up becoming, and I was just going to call it “WebNotes”.

But when it grew and I decided it was worth sharing publicly, I decided to give it a real name. That’s how “FrankMD” was born. But it’s not enough to just give it a name; now you have to go rename EVERYTHING inside the code, strings, documentation, etc. It was in this commit: a solid 700 lines of modifications.

Now “does it work??”

I’m writing this article at 5 PM on Sunday. At 7:25 PM on Saturday, I sent it to some friends on Discord to test, innocently thinking I’d already reached version 1.0.

And there I went, with a fever and needing to lie down, but wanting to reach 1.0!!

The issues themselves I killed quickly, but Syntax Highlight I didn’t know was going to become hell in my life. The first version came out quickly, in this commit

Don’t Reinvent the Wheel (duh)

Yes, it’s the most obvious cliché of all, but even I make this mistake. As I said at the start, my expectation for this project was to be little more than a Web “Notepad” for my notes, and it ended up becoming the main editor for my blog.

The text editor part started with the first thing that works first: just a plain textarea.

But on Saturday I wanted to add a line counter column that takes line wrapping into account, control over that wrapping, typewriter mode always centered in the middle of the screen, and sync between the text editor scroll and the preview.

Not counting all the dialog boxes that add things to the text, like images, videos, code and tables. Not counting the shortcuts for things like bold, italic. Not counting auto-save routines, line count stats, current line, etc.

For those who don’t know, the problem with syntax highlight is that you can’t edit directly on the colored text (because it has hidden tags that format the text). One way to solve this is to duplicate the text block: one in the background, which is the raw text and you can edit however you want, and another formatted and colored block with HTML exactly on top, which you don’t touch, but it updates with whatever is edited. That’s how Claude did it.

But there are a LOT of “edge cases”: alignment problems due to wrapping, problems with size and position to adjust the auto-scroll sync. I spent hours and hours fighting with this. Look at the kind of problem I was dealing with:

Everything got resolved when I decided: enough. I’m reinventing the wheel. Obviously this problem has already been solved. So I asked Claude to remove all the logic tied to the custom textarea and replace it with Code Mirror, which is an open source component for a code editor, with support for line counting, syntax highlight, and much more!

It wasn’t just those 5 commits; those were the main ones. They took a solid 3 hours or more to stabilize that code, plus multiple small adjustments over more commits. A good 6,000 lines of code modified. It’s the kind of refactoring that, in a real project, would have taken the whole week and still would have come out with multiple bugs and side effects. But for this, the LLM is also very good. Big refactorings, especially if you’ve prepared a good automated test suite, go quite well with LLMs.

This is the kind of thing that happens in real projects. I can’t blame the LLM, it’s entirely my fault for changing the project requirements: originally there wasn’t going to be syntax highlight!

Tests, tests, tests

The main recommendations for how to do vibe coding are the same as how to program the old way:

• Be careful with architecture choices (yes, it takes experience)
• Build small features one at a time, each with the right abstractions and immediately with tests
• Refactor from time to time to consolidate or abstract code into the right domains, adjusting and adding more tests

If you follow those 3 items, it already helps a lot. It’s not magic: all of these are tools that give feedback to the LLM so it knows what to do.

If you’re the kind of dumb customer who only says “oh, it stopped working” — you’re no good at talking to a programmer, and I’m sorry to say that LLMs won’t help you either:

• WHAT stopped working?
• Did everything stop opening?
• Did just one element disappear?
• Does the element appear but clicking does nothing?
• Can you click, but an error message appears?
• What message exactly?
• What did you do before that?
• Did you change anything from the previous test to this one?
• etc

ALL OF THIS and more has to be in the next prompt for fixes. Including server logs, command lines to reproduce the error, and whatever evidence can be produced, to make it easier to understand the error.

There’s no such thing as “it stopped working” — your head is the thing that never worked.

One thing I really liked about Claude was when a bug appeared that hadn’t existed before, but I didn’t know if it was from the last commit or from 3 commits back. And instead of just guessing at random things, Claude itself offered to manually do the equivalent of a git bisect and kept offering me one commit at a time to test and keep going until we found which one introduced the bug:

Optimizations and Checks

Throughout the project, 2 or 3 times I specifically stopped to check things like memory leaks and security holes. Even being a personal project, I didn’t want very obvious holes.

Be warned that Claude is a champion at scattering setTimeout calls in JavaScript and never calling clearTimeout on anything!

It’s not much, but I had more than one commit to delete “Dead Code”: code that, during a refactor, was moved to another module but forgotten at the origin. LLMs will always leave dead code behind, because they have no way to look at 100% of every file 100% of the time. Eventually it misses one thing or another. And it won’t cause any obvious bug. You have to prune it gradually, after each refactoring.

When I started really testing with long texts, I noticed the interface had “lag”, kind of slow. Two things seemed to be causing it: the statistical count of how many words and characters the text has was very aggressive.

Again, “it already works” … but slow …

Worse: when I first asked it to create the text editor, I didn’t say exactly how I wanted the “auto-save” to behave. So it did it in the most conservative, aggressive and slow way of all: save everything on every keystroke!!!

There were several other things like not updating the HTML Preview if it wasn’t open and things like that.

LLMs won’t make performant code right off the bat; you need to tell them how!

Performance is always a trade-off. For example, if I go as fast as possible, the rest may end up super insecure. If I crank up security too much, usability collapses. If I over-optimize, maintainability collapses. There’s no way to make everything perfect: one will always negatively affect the other, and only an experienced programmer knows how to choose (or goes through trial and error until getting it right for the specific case).

Again, it’s not the LLM’s “fault”; this is normal in any project and we have to keep adjusting these things all the time.

Now really, reaching the end

There’s still plenty of technical debt left. The app_controller.js file still has more than 2,000 lines. And this is AFTER I spent a solid 3 hours refactoring just this one. Now there are roughly 28 JavaScript controllers, but before there was only a single file.

Besides that, LLMs can’t be consistent all the time, especially when the project code grows a lot. There was one type of feature it always did server-side, but later decided to do client-side. I had to interrupt and say: “hey, why?”, and then it redid it the right way:

Any code written with zero human assistance ALWAYS comes out at the worst possible code quality. With my assistance, I think the code is at least minimally presentable. You can check for yourselves at my GitHub repository.

I recommend digging through the commits: at least one thing LLMs do well is document the history of what was modified in each commit, which makes it much easier to search later.

There was one bug where Claude got stuck in a loop, trying and failing, to figure out when my CSS broke out of nowhere. It changed something and asked me to test; I restarted the server, did a force reload, and nothing: broken.

It was Saturday night; I was tired and a bit feverish, but then the penny dropped. I had done something dumb earlier. I put “notes/” in the .gitignore, which was the directory where I was creating test Markdown files in the editor and didn’t want to commit.

However, that also started ignoring directories like app/views/notes/..., for example. And neither I nor Claude knew this, but when I asked it to search, voilà:

Tailwind build obeys .gitignore, so it was screwing up the CSS compilation. 🤬

When we fixed that, it started working, but by then 2 hours had gone by!

As I said, you have to stay sharp on the errors, watch why the LLM is doing so much trial and error, doing and undoing in the same place and getting nowhere. This isn’t uncommon and can eat up half your day if you’re not paying attention.

Conclusion

When I started this Friday morning, I imagined:

“Ah, a full day of work should be enough.”

And as always, every estimate is an underestimate. It’s already Sunday, end of the day, and only now do I think I’ve gotten close to version 1.0. It still isn’t perfect, but I think it’s at least minimally presentable.

Remember I did this at my pace of extreme hyperfocus, when I barely blink, my eyes dry out and I start to see everything blurry, and even my head starts to hurt a bit from staying so tense, concentrated on a single thing. It’s not a state I recommend.

Even those 30 hours over 3 days, the right thing would be 30 hours spread over at least 5 days.

If I had let Claude do everything on its own, without checking anything, it would have taken half the time and, in the end, I’d have code “that works”, but bizarre, with a single CSS file of around 2000 lines, a single JS file of around 8 thousand lines, everything with ZERO tests, and everything looking like a house of cards where you touch one small thing and break 10 other things without knowing why.

Overall, it was a good experiment. Knowing how and where to use them, LLMs are exceptional. As I said at the start, without the LLM I would have taken more than 200 hours to do it alone. Even going through several rough patches along the way, I only took 30 hours. So the LLM really did make me work at least 6 to 7 times faster, maybe more.

Finally I feel we’ve reached the elusive “10x Developer”. All you need is to be a senior first.

To wrap up, a mini-report from Tokei:

Breakdown by language:
┌─────────────────┬───────┬────────┬────────┐
│    Language     │ Files │ Lines  │  Code  │
├─────────────────┼───────┼────────┼────────┤
│ JavaScript      │ 137   │ 26,621 │ 19,922 │
├─────────────────┼───────┼────────┼────────┤
│ Ruby            │ 60    │ 8,449  │ 5,883  │
├─────────────────┼───────┼────────┼────────┤
│ YAML            │ 10    │ 3,477  │ 2,867  │
├─────────────────┼───────┼────────┼────────┤
│ Ruby HTML (ERB) │ 33    │ 2,927  │ 2,634  │
├─────────────────┼───────┼────────┼────────┤
│ CSS             │ 29    │ 1,619  │ 1,297  │
├─────────────────┼───────┼────────┼────────┤
│ Markdown        │ 1     │ 716    │ -      │
├─────────────────┼───────┼────────┼────────┤
│ HTML            │ 5     │ 369    │ 329    │
├─────────────────┼───────┼────────┼────────┤
│ Dockerfile      │ 1     │ 88     │ 46     │
├─────────────────┼───────┼────────┼────────┤
│ Other           │ 6     │ 145    │ 76     │
└─────────────────┴───────┴────────┴────────┘
The codebase is primarily JavaScript (60%) and Ruby (17%) by code volume.

So, roughly 33 thousand lines of code. This is a SMALL project! Big projects are 10 times bigger, with more than 300 thousand lines of code. I’d guess that at an iFood or Mercado Livre, with all the subsystems, it’s more than 1 million lines of code. Adjust your expectations!

I’ve been saying this for years: I LIKE LLMs, I use them a lot since they came out, and I’ve never had a problem with them. My problem is calibrating the UNREALISTIC expectations of “normies”, non-programmers: you’re never going to get an app like this with a handful of prompts without knowing exactly which components and techniques to use underneath.

This time, I decided to tackle another little problem that had been sitting on my todo-list for a while: having a self-hosted note-taking web app (to put on my home-server. Over the years I’ve used everything: Evernote, Notion, Obsidian, among others, but none of them really pleased me. So why not build my own exactly the way I want?

I’m going to split this topic into 2 articles: in this first one, I’ll talk exclusively about my application; in the second one, I’ll talk about the process of vibe coding with Claude. Spoilers: it took 3 full days (roughly 30 hours). 137 pull requests! Wait for the post-mortem.

The Problem

Overall, this is just me being fussy. Keeping notes shouldn’t be complicated: create a directory on your machine, open LazyVim, VSCode or whatever, done: edit the Markdown as pure code and be happy.

Anyway, recently I was just using Obsidian on a shared directory in my Dropbox. That way I could access it from my phone if I needed to. But it’s a pain because if I’m on another PC, I need to have Obsidian or some other editor installed. If I have plugins, I need to install those too.

In practice any other text editor would do: they’re just text files, but the inconsistency of having something different in each place, all misconfigured, demotivates me from writing.

In particular, I was using Obsidian and, later, LazyVim to edit my blog posts, which are also in Markdown. And here I have a bigger set of problems:

• there’s a specific path and Hugo frontMatter; I have to be careful with the title to generate the right slug, etc.
• there are images/screenshots that I keep manually uploading to S3, grabbing the URL and pasting into the post
• I avoid using tables because “drawing” tables directly in Markdown is a pain
• I keep Google Images, YouTube, etc. tabs open so I can keep grabbing links, alt-tab, and paste
• there’s no good preview, so I let a lot of little errors slip through (duplicated image url, for example)
• there’s no good spellchecker, so lots of little typos get through that I then have to keep fixing

I wanted to solve all these problems in one place: a beautiful editor like Obsidian that would fix all these inconveniences and let me focus only on my text.

FrankMD

Frank Rosenblatt was the creator of the Perceptron in 1957, which gave rise to modern neural networks. Since I built this editor entirely with AI, I decided to pay tribute to Frank.

So I named it Frank Markdown or Frank Editor (fed). This editor has all the basics you would expect in any editor:

• syntax highlighting
• HTML preview
• markdown formatting helpers (Ctrl-B for bold, for example)
• customize font family and size

But then I started adding some things specific to my personal taste and/or that would make my life easier for editing my blog posts. Let’s go.

Image Insertion

This is an important part for me. When I write posts about code, I’m always taking screenshots of my terminal or editor to illustrate. The process is a print screen that sends it to ~/Pictures/Screenhots. I open Nautilus (I’m on Linux) and run a little script I made to upload that image to a bucket on my AWS S3. I copy the generated link and add it to my post.

It’s “a little annoying” doing this every time. On top of that, sometimes I’d like to resize the image to make it more optimized. But that means going to the terminal and running magick screenshot.png -resize 50%, which is another manual step.

So I decided to make a dialog box with several options for this.

By default, I can configure a main images directory on my PC and it will display a grid limited to the 10 most recent. Or I can filter by name. There’s an option to link directly (in case I want to upload the image along with the html) or to serve from S3 and it will copy the generated link automatically (which is what I prefer).

Same thing for images on the Web (Google, Pinterest), where I can choose to download, resize and upload to S3:

And the link will be in the expected format ...amazonaws.com/frankmd/...jpg. All pretty, easy and practical! For me, this helps a lot!

It’s an extra, since I couldn’t skip integrating an AI; the last tab is for Nano Banana:

Nano Banana is really impressive. Look at what this prompt generated:

Even more important: I can see the preview immediately (ctrl-shift-v) to know whether I inserted the right image (something I used to screw up a lot editing in NeoVim, without seeing it):

For me, these image insertion helpers are the ones that are going to help me the most when editing my blog posts.

Markdown Helpers

Simpler than an image is embedding a video player. I can either pass any link with a random .mp4 file, or I can directly search YouTube and embed it properly:

It will insert the correct HTML, <embed ..., with the right URL and title.

I also built an easy way to add and edit tables:

And it will generate the Markdown just right. Never again do I have to sit there editing that garbage by hand to readjust columns or rows.

| Cabeçalho 1 | Cabeçalho 2 |
| ----------- | ----------- |
| Linguagem   | Nota        |
| Ruby        | 10          |
| JS          | 5           |
| Rust        | 6 😂        |

But if you want to do it manually, you can also do that:

I added this mostly because it would be weird to leave it out, but in general I prefer to write code blocks manually. But there it is for anyone who wants a visual mode.

Then, of course, it had to have an easy way to edit Emojis 😆 (ctrl-shift-e):

But what nobody would expect is that I also offer support for Emoticons!! ☆:.｡.o(≧▽≦)o.｡.:*☆

Customization

Speaking of Emoticons, that reminds me that I wanted this editor to be infinitely customizable. For example, of course I added internationalization and it has Japanese!!

And other languages too; go poke around later. It has Portuguese too!

“What’s the best font for programming?”

Who knows! That’s why I added several to choose from:

Since I use Omarchy, I also wanted my editor to support exactly the same themes:

In the README everything that can be customized is documented. You can customize many things through environment variables and also through a .fed file, automatically created in the directory where you open this editor. There’s plenty, so I recommend poking around.

Focus on the Text

Obviously I included a “Typewriter Mode” that keeps my cursor closer to the middle of the screen and centers the editor, while hiding all distractions.

I normally prefer TUIs, but I have to admit that using a browser has the advantage of giving me access to extensions. The most useful one, without a doubt, is Grammarly, for any language:

But beyond that, I added an “AI” button in the toolbar (which is configurable via ENVs and a config file — read the README). And it does this:

It sends to your LLM of choice (local Ollama, OpenAI, Anthropic, OpenRouter, etc.) with a prompt telling it not to rewrite your text, only to check grammar, spelling and errors of that kind. Then I return a “diff” interface to compare and accept, edit or cancel.

I think Grammarly alone is already enough, but it cost nothing to build this one, so I built it.

Hugo Blogging

This blog is made with Hextra/Hugo. That’s why I wanted to already bake in an easy way to create new articles in Hugo. It starts right from the option to create a new document:

And it already creates it with the frontMatter, automatic slug generation, timestamped subdirectories and all the rest:

Like I said, this editor was built mainly to make my own workflow easier.

Self-deploy vs CLI

Originally I thought of this project as a Notion-light or something like that. A web editor that would deploy to my home server and serve only as an organizer, a place to put notes bigger than a post-it (like the history of my Linux configurations, step-by-step commands, etc.).

But as I kept building the app, more and more it grew and started looking like a desktop app. But by then it was too late. I wanted to use Ruby on Rails 8 (to shake off the rust and because I like it). Only halfway through did I think I should have done it in Node, because now I can’t embed it in an Electron app anymore.

Fine, the idea is to be a Markdown editor for programmers. Every programmer has Docker. So I wrapped it in a little shell function and I can just land in any directory on my Linux and type:

cd ~/Projects/akitaonrails-hugo
fed .

Read the README of the project to learn how to add it to your .bashrc. And doing it this way, it will work on my Omarchy exactly like any other native app:

In reality, I’ll use both ways: as if it were a native app. And as self-hosted, to organize a central place with my notes — which will be separate from where I organize my blog posts. So it really does have multiple uses.

Conclusion

I’ve summarized the features; there are several little things you’ll discover as you start using them. Everything is clickable; if you click on the app title, there’s an About:

Right-click on top of selected text has a formatting submenu; Ctrl-I works for italic; Ctrl-H has a basic “find and replace”; there are a bunch of keyboard combinations to access things.

Oh right, I didn’t even mention that you can quickly search files by name or by content using Ctrl-P and Ctrl-Shift-F:

To make sure everything is working, this very article was written entirely with my new editor. And I’d say that, if it isn’t perfect (for me), it’s at 99%. Feel free to open Issues with bugs or features that would make sense — remembering: the idea is to stay away from replacing VSCode and having EVERYTHING. I need it to have only ENOUGH.

Considering it cost me 3 days of work and that I didn’t write a single line of code, the result is impressive. Yes, text editors are very well documented applications with tons of training material — that’s why Claude managed to build it without trouble. Don’t get too impressed. That said, yes, I wanted a custom editor for my personal use, one that would be beautiful to use, and it managed to deliver, so I’m satisfied.

Don’t miss PART 2 for the details of how this was built.