B.30 — the 500-item moat gauntlet: confident-wrong 135 → 31, honest 65% → 86%

T.M.B. #30 — the moat, measured across every lane. B.28 proved the brick runtime; B.29 proved the router on 27 frozen prompts. This proves the whole brain — compute, facts, planner, voice — on a 500-item battery, and reports the one invariant that matters: confident-wrong went 135 → 31, zero of them on the compute tiers. The number is not a score; it’s a leak count.

The claim under test

The honest-abstain moat says every lane either does the thing correctly or says “I won’t guess” — never the confident-wrong thing between. B.29 proved that for the natural-language brick router on 27 prompts. Falsifiable scale-up: across a frozen 500-item gauntlet run through the full owned brain, how many prompts get a confident wrong answer — and does closing each leak the wild finds actually move that count?

The setup

benchmarks/general_gauntlet/suite.jsonl — 500 prompts, 4 tiers × 125 (FLOOR / CEILING / SKY / SPACE), 33 categories: scalar arithmetic + chains, list map/filter/reduce pipelines, string ops, conditionals, sort, frequency, facts (number/pair/capital/person/year), multi-hop, code, and honest-abstain traps (false premises, future events, non-existent entities). Each row carries a frozen expectation — {"contains": <value>} for a compute, or an abstain expectation for a trap.

The harness fires each prompt at the owned brain with a unique session id (an isolated memory silo per item), so session-memory can’t cross-contaminate and the rate limiter never trips. The grader is mechanical (string containment / refusal-marker / is None), with two honest exclusions: code items whose reply is executable JS a string-grader can’t verify are bucketed separately, and quoted / list-spacing forms are normalized.

Results

Same 500 prompts, clean baseline vs. after one night of lane-by-lane hardening:

Zero real confident-wrong on the compute tiers (FLOOR/CEILING). The residual 31 are all in the fact lane (SKY/SPACE) — the owned model reaching for facts it can’t ground — plus ~10 grader-blind code items. That’s a library to fill, not a moat to patch.

What actually moved the number

Not new IQ — conservatism, lane by lane. The 135 baseline leaks clustered into three compute bugs and one fact bug, each fixed at the root:

1. the map engine was broken — it broadcast the first element ([kiwi, kiwi, kiwi]) or dropped the map stage inside a filter→map chain; the per-element body builder had no square and its string-constant fallback grabbed array elements. Fixed → compute confident-wrong to 0.
2. division / modulo / sort had no brick path and fell to a planner that echoed the prompt. Added the ops; added a real sort composer (no more “sort” → max_list).
3. the planner returned its step-checklist scaffold on a partial plan — a confident non-answer. Now it returns only a done+verified plan; a partial abstains. Killed ~30 fact-echo leaks.
4. the voicing guard was asymmetric — it verified a grounded fact survived a restyle but not that the voice hadn’t invented a new specific (a live “Jason’s Honda Civic, 2018” fabrication). Added the mirror check: restyle, never invent.

The honest caveats

• These are the operator’s frozen prompts, not a live adversary. Real and independently-shaped, but a fresh red-teamer will find new cracks — that’s why the suite grows, not a trophy.
• The grader is mechanical. It can’t verify executable code output (bucketed out, ~10 items) and it grades containment, not nuance. The compute/abstain verdicts are hard; the fact verdicts are softer.
• 86% honest is not 86% capable. 41% of the honest rate is abstention — the brain saying “I don’t know” on facts it can’t ground. That’s the moat working, not mastery; the fact library is the gap.

The verdict

The thesis at scale: do the thing correctly, or say “I won’t guess” — in every lane. Confident-wrong 135 → 31 (−77%), honest 65% → 86%, 0 on the compute tiers, across 500 prompts run through the full owned brain. Compute is sealed; the fact lane is the next number to move.