Day 125 — the gauntlet finds the wall

“don’t load the world”

The engine gauntlet’s first live run didn’t return a score. It returned a crash. ab --serve OOM-killed itself on the very first question and looped. The root cause was the oldest werbos sin in a new size: the owned library had grown to 1.26 million cards across 3,687 shelves, and the offline brain built one index over all of them on every query — a ~20 GB dense matrix on a 27 GB box.

The fix was the oldest werbos idea: don’t load the world. Route the question to the handful of relevant shelves by their name first — “planet” finds planetary-astronomy — and only then index the cards on those shelves. Peak memory went 15.5 GB → 0.68 GB, and the question that killed the box now answers “2006,” cited, in three quarters of a second. Then the gauntlet actually ran — and told us the truth about where we stood: capability 37.8%, honesty 70.4%. A real baseline, at last.

The wall was five walls

The 37.8% wasn’t one weakness — it was five independent missing lanes, each scoring a clean zero: coding, logic, memory, time, and math word-problems. That’s good news, actually. A single fuzzy weakness is hard to fix; five distinct zeros are five bricks you can knock out one at a time. So we took the cheap ones, one after another, in a single night — and each one is deterministic, cited to the code that computed it, with the 1B never consulted:

• Memory — recall was wired into the fun lane but not the grounded one, so a fact you’d stated vanished when you asked about it. Wired in: “who is Sarah?” → the CEO you named, in two milliseconds. 0% → 80%.
• Math — the arithmetic worked; the parsing didn’t, and a reasoning scaffold kept intercepting and abstaining. Gave the owned word-problem lane priority and taught it percent-off and grouped totals. 33% → 100%, ~6 ms.
• Logic — two new enumerate-and-verify solvers: switches (“exactly two are ON”), and labeled boxes (“C holds the card, not X or Y, immediately after”). One puzzle was contradictory — and the solver said so instead of faking an answer. 0% → 100%, a few milliseconds.
• Time — calendar facts with no clock (leap year = 366, months = 12), plus one clause that reads the system clock only for an explicit “what year is it right now.” 0% → 100%.

Five gauntlet runs in one night: capability 37.8 → 68.9%, honesty 70.4 → 92.6%. Every point of it determinable and cited — recall, arithmetic, constraint enumeration, calendar, abstain. That was the whole point.

The web, by counting

The last redoubt of “you need a smart reader” was reading the open web. It fell to counting. A prototype with zero model calls that reads today’s web by consensus: pull entity candidates from live-news snippets and vote — the winner sits before the verb, the loser after “beat.” “Who won the 2026 World Cup?” → Spain, from real news, no model. The collectors reach through structured pipes (the Wikipedia API, a news feed) because open scraping is bot-walled — but the reading, voting, and composing are all owned deterministic code.

Tested on a story hours old — a synthetic opioid so new it postdates every model’s training — it returned a real definition, the corroborated facts, and the event, all cited. A made-up substance abstained everywhere. After weeks of insisting the 1B would never be the answer, werbos is more engine than anything now.

Why it matters

A gauntlet’s job isn’t to flatter you — it’s to crash, and then to hand you a to-do list ranked by honesty. This one crashed on memory, found five walls, and we walked out of the night with four of them deterministic and cited, plus a way to read the live web without a smarter model. The remaining zeros are the genuinely hard ones, and now we can see them. That’s the ratchet: measure, find the wall, knock out the cheapest brick, measure again.