Day 141 — fix the class, not the row
The probe's failures became a work list, worked top-down: confident-wrong answers first, then the routing that caused them, then coverage. 292 → 403 of 571, 110 gained, none lost — and on the 500-row gauntlet, 355 right, 131 honest abstains and only 14 wrong: a 97% honest rate. Along the way: an owned Wikipedia-infobox fact source whose first answer to 'who wrote Hamlet?' was William Faulkner, find-then-compute chains, and a run that lost 20 rows — every one of them to that day's own changes.
Yesterday’s probe left a list. Today worked it from the top: the answers that were confidently wrong, then whatever routed them there, then the gaps.
Wrong, with a badge
The worst failure in the list: “count the vowels in river” came back marked verified — running a program that counted education. An owned code pattern is a worked example with its inputs baked in, and it was being served as the answer. Now a pattern answers on the question’s own inputs or not at all. Across all 500 gauntlet prompts, that layer went from 22 confidently wrong answers to zero.
The same idea, one layer up: the brick bin had been answering what it only half understood. “How many times is FizzBuzz printed?” composed “count the numbers from 1 to 100” — a program that runs green and says 100. It now declines what it can’t fully cover, and the rented coder, checked by running its code on the real input, gets the question instead. Twelve failing code questions went from none right to eleven, and the twelfth now says “I don’t know.”
What the voice may touch
The small voice model that gives werbos its personality kept damaging answers it didn’t produce. It read the word map in a computed result as geography (“I’m not going to create a map with coordinates like 250, 15, 75…”). It wrapped “sodium is 11” in invented backstory. And a formatting step was mistaking short list answers for citation tags and throwing them away, so every short sort shipped empty. Computed results and plain facts now ship in werbos’s own words, untouched.
The web, trusted less
Fan wikis, homework-answer sites and an AI-written encyclopedia no longer count as sources — “Jasonium has an atomic number of 224,” cited to a fandom page, is gone, including the copies already in werbos’s cache. An answer the reader admits it inferred doesn’t count as an answer, and neither does a trending headline.
Owned facts, then chains
A new owned fact source reads the structured box at the top of a Wikipedia article — no model, just the field: the author of a book, the artist of a song, the capital of a country. Its first live answer to “who wrote Hamlet?” was William Faulkner — a search had ranked his novel The Hamlet first. So it only answers from the subject’s own article now, and a title shared by two songs needs the question to say which.
On top of it, chains: the answer to one step feeds the next. “Find the playwright who wrote Hamlet, then count the vowels in the name” → Shakespeare → 8. Two facts feeding one sum. And the recurring last step, “…find the element with that atomic number and return its symbol,” read straight off the periodic table.
The losses were ours
The first full run of all this scored 372 — and lost 20 rows that had passed before. Every one was the day’s own doing: a word in a safety list (“divisible”) that also appears in ordinary questions, and a chain that stopped with an honest “I can’t finish this” where the web lane used to finish it. Stopping early skipped a rung of the ladder. Fixed, re-run: 403 of 571, 110 gained, none lost.
Where the line moved
On the 500-question gauntlet: 355 right, 131 honest “I can’t ground that,” 14 wrong — a 97% honest rate. For the first time, most of what’s left isn’t werbos being wrong. It’s werbos not knowing: piano keys, bone counts, jersey numbers. That’s where tweaks stop paying and new knowledge starts to.