Day 124 — the engine does the work

“the engine does the work”

The gauntlet came in from ChatGPT — ten questions, adversarial, meant to break the grounded chat. It scored 1 out of 10. But the glass-box told the real story: for most of them the engine had retrieved the correct card — “Pluto reclassified as a dwarf planet in 2006,” “the assassination of Archduke Franz Ferdinand,” “sunlight takes eight minutes twenty seconds” — and then refused to say it, because a grounding floor set to reject one shallow guitar card was rejecting every good answer too. Crushed with the safe on.

Grounding that holds

Dropping the floor flipped it — 1/10 to 7/10, on questions neither of us had seen. But the fix exposed the next failure: grounding on a card that scored high but was off-topic — “Moon vs Pluto” answered from a bioaerosol card, “capital of Australia” from a parenting card. Score isn’t relevance. A relevance gate — the card must share the query’s named subject — turned those confident-wrong answers into honest ones, and let the right card (Canberra) through. And the faithfulness guard got smarter: it stopped demanding every fact-word survive verbatim (which rejected “has a limit of two terms” for dropping “limited”) and started checking anchors — numbers and names strict, framing words loose. A looser 1B mouth can paraphrase all it wants now; it just can’t lose or fight the fact.

Memory

Then the harder attack: state a fact, chat, ask it back. “My company is Acme, its CEO is Sarah, founded 1987.” …a while later… “Who is Sarah?” — and it said Sarah Vaughan, the jazz singer. The chat was stateless per turn; everything you told it evaporated. So the conversation became a tiny, retrievable card library: keep the user’s own turns, retrieve over them, ground on them before the global hub — the user’s words about their world outrank any card. It answers the segment now (“You ghostwrote it for Richard”), not the whole sentence echoed back, and it catches “what did I just fix?” on the recency of the ask. Constant memory — retrieve the one fact — not linear, holding the whole conversation in a giant context. The frontier’s way, inverted.

Reasoning

Last, a logic puzzle — four boxes, statements, “exactly two are true, where is each object?” No card, no lookup, and the 1B did what a small model does with hard reasoning: it hallucinated a confident, gibberish proof over seventy-seven seconds. But the space is finite. So the honest move isn’t to generate a proof — it’s to enumerate and verify. The solver tries every arrangement, keeps the ones that satisfy every clue, and reports the truth — including “this has seven solutions, it’s underdetermined” or “no arrangement works, it’s contradictory.” Both puzzles ChatGPT handed over were broken — one underdetermined, one contradictory — and the solver caught both, in milliseconds, where a confident model would have faked an answer to a broken question. Then the parser grew: ordering puzzles, odd/even, adjacency, “not in Room 2 or 4,” “exactly one of these is true.” Each new evil puzzle, absorbed.

Why it matters

The through-line, across all of it: the model gets out of the way, and the engine does the honest work. Retrieve, verify, enumerate — never generate-and-hope. And that’s not just tidy; it’s the shape of the fight against a 30B. You don’t beat a bigger model by out-generating it. You out-structure it: win the clauses where the answer is determinable — logic, code, grounded facts, remembered conversation — provably, and say “I don’t know” honestly on the rest. On both of those axes the engine does the work, so the 30B’s size is neutralized. And the killer axis is the long conversation — the frontier holds context linearly and hits a wall and forgets the middle; werbos retrieves one fact and keeps walking. A small honest engine, built to beat a big confident one.