Day 123 — the mouth stops guessing
Started as a latency question — why does the 3B take 100+ seconds when the engine parses in 30ms? — and ended by inverting the contract: the engine owns what's true, the 3B is just the mouth. It no longer authors answers from the prompt and ignores the cards; it voices what the cards and the reasoning already decided. Grounded riff, a bounded reasoning lane, a glass-box that shows its own cards and their scores, and a floor so a weak card can't pose as truth — shipped and live on SR8 the same day.
“the mouth stops guessing”
The question that started it was about speed: why does the 3B take 100+ seconds after it’s warm, when the engine parses the question, looks up the cards, and decomposes the reasoning in under 30ms? The answer is that those are different physics — lookup is one pass over an index; generation is hundreds of sequential passes over billions of weights. You can’t make generation as cheap as lookup.
But chasing that led somewhere better than a speed fix. The real disease was never the latency. It was that the 3B was authoring answers from the prompt and ignoring the cards — confident, fluent, and accountable to nothing. So we inverted it.
Don’t send the model the question. Send it the answer.
The riff mouth used to get the raw question plus a weak card and had to figure out what’s true and phrase it. It’s bad at the first job and slow at both. Now the engine decides what’s true; the 3B only decides how to say it warmly (103fcc7). The seed got a relevance floor, which killed the bug that opened the whole thread: “could Obama run a third term?” had been seeding off a random running-nutrition card and answering with marathon trivia. No relevant card now means riff on the question honestly — never a junk card dressed as the answer.
A reasoning lane that grounds and proves
reason_qa puts reason_tree on the live path for a bounded set of shapes (103fcc7). Ask “can a
president serve a third term?” and it decomposes to one atomic fact, grounds it in a card, and composes
“No. The U.S. presidency is limited to 2 terms, so a third term isn’t allowed” — with a Merkle proof
and a citation, in about 200ms. The 3B never touches it. Everything it can’t shape-match falls through
honestly. Live testing on SR8 immediately earned its keep: real cards say “two elected terms,” not
“2,” and the digits-only parser was silently gapping on the word (478a8a5).
Grounded, yet conversational
The riff didn’t need to lie, and it shouldn’t abstain either. So now it resolves the true answer first, states it, then riffs on top (a6763b0):
The two-term limit thing? Yeah, that’s a real deal. The 22nd Amendment says so — no third term for any prez. But imagine if someone actually wanted four terms…
That fixed a subtle failure the pure-vibe version had: it used to say “Trump can’t, but Obama could!” — inconsistent, because both are term-limited by the same rule. The engine now owns the fact; the mouth owns the vibe. And with the output capped (44fe0c8), the riff dropped from 43 seconds to about eight, without losing the warmth.
The glass-box shows its own cards
The debugger used to say “grounded” and make you trust it. Now the trace shows the cards —
[library] score · snippet — and the exact fact the answer was built from (6fb601d, fac054f). When
the reasoning lane grounds an answer, it shows the true source card, not the misleading raw-query
neighbours. That made the next bug diagnosable at a glance: ask how to fix a flat-sounding guitar and the
trace showed it had grounded on a [guitar] 17 · reference note card — a weak match that missed “flat”
entirely. So we added a floor (90d334a): a riff states a card as truth only above score 25. Trump’s
term-limit card (41.5) still grounds; the guitar card (17) drops to an honest free-riff instead of a
confident, shallow one.
The AHA
The thing worth keeping is what the operator said watching it work: “we didn’t train the cards on this.” The 3B does not reliably know the 22nd Amendment. A card does. The model just voiced it. So teaching it something new isn’t a training run — it’s a text file. Add a card, and it knows the fact tomorrow. No GPU, no retrain, $0. The 30B bakes knowledge into weights and hopes; werbos streams it from a library the engine reads and the glass-box shows. A librarian who can hand you the page beats the guy who memorized the encyclopedia and makes up the parts he forgot.
A followup resolver also landed (103fcc7) — it reads “well, he is” as he → Trump, a challenge that
disputes the prior answer and raises a needs_verification flag. Built and tested, waiting on the
verify lane it points at. That’s tomorrow’s thread.