Day 48 — 'outside the model.'

“outside the model.”

Yesterday’s lesson came from the live 1Password door: a model proposed, a model graded, a human signed off, and all three were wrong together — only the server, outside every model, was right. Today is that lesson turned inward. Four fixes, and each one moves a judgement OUT of the model that’s being judged.

The self-model stops flattering its magnets (c35418f). reflect’s “I’m reliable at…” came straight off the soft self-grade, so a greedy skill that grabbed a request meant for someone else — satisfied=true on a hijacked prompt — read as a strength. Now every satisfied action is re-routed through the deterministic skill-router (high-precision-or-abstain) and only counts as a clean win if it was actually on-target. best_skills uses the routed rate; the magnets fall out and get named — “Greedy — grabbing work meant for other skills: what_if (100% misrouted).” The routing honesty that lived only in the dashboard finally reached the self-model the Mind page reads.

The autopilot routes outside the model (11b77f0). The diagnosis surprised me: the deterministic router was never the greedy one — it abstains correctly. The magnets came from the LLM predictor, the small model forced to pick a skill for every prompt. So the fix wasn’t to tighten the router; it was to put the router in front of the model. When it confidently routes to an allowed skill, use it — free, deterministic, no magnet — and fall to the LLM only on what routing can’t decide. werbos’s own north star, applied one layer deeper.

A grade no model can talk its way past (2753cfb). The unstuck loop’s “hard” check was a program the model wrote — and a model-written check shares the model’s blind spots (it certified trailing-zeros=5 when the answer is 6). The fix is the only truly un-bribable tier: a curated bank of (question → TRUE answer), authored by a human, checked deterministically, tried before any model writes or runs anything. It catches the exact confident-wrong answers the soft judge and the exec check both rubber-stamped — and it abstains on anything not banked, so growing the bank is how werbos accumulates verified knowledge it can grade itself against. The general analogue of yesterday’s live-CTF door.

AHA: every grader werbos trusts now has its feet outside the thing it’s grading — the router oracle, the live server, the human-authored answer key. A judgement made of the same stuff as the work can always be fooled; a judgement anchored outside it can’t. That’s the whole week in one sentence.

One shape for the evidence (44b2519). Last, the plumbing the rest will stand on: ExperienceStore now adopts soma.capture — the append-only store that was carved from it weeks ago, used back. With the sibling installed it logs through capture; without it, a byte-identical fallback, so the format never forks and the container deploy doesn’t blink. Every experience row now carries a surface tag and a ts, which means autopilot, mythos and dogfood all write one shape — and a future learn can finally read across them instead of one silo at a time.

Four commits, ~19 new tests, ~1900 green. No new feature you can screenshot — just every place werbos judges itself moved its standard outside its own head. The model proposes; something that isn’t the model decides. That’s the spine the psychology work has been reaching for all along.

And then the loop that LEARNS from all that honest grading. The arc had one piece left: the slow loop that trains werbos’s routing policy. It was already built and ticking inside werbos serve (champion/challenger — train a candidate on past experience, promote it only if it beats the incumbent on held-out data), but it scored skills on satisfied — the soft self-grade. So a magnet that grabbed a prompt meant for another skill got satisfied=true, earned routing credit, and taught the policy to route toward the magnet. The learner was diligently compounding the exact dishonesty the rest of the day tore out.

Honest fuel (4a5e2a6). A row now teaches the policy only if it was satisfied AND on-target — satisfied and not a confirmed misroute by the router oracle, with a known-answer FAIL never earning credit even when the soft judge was happy. Both training and held-out scoring are gated this way, so the gate optimises on-target satisfaction. Production gets it free — the trainer already runs; it just drinks clean water now.

AHA: an automated learner is only as honest as the signal it rewards. We’d made the grade honest everywhere and still left the learner sipping the soft judge — quietly compounding the error. Closing a loop isn’t wiring it up; it’s making sure the thing it optimises is true.

The gate becomes a primitive (b79189c). That champion/challenger gate was too good to leave locked inside autopilot, so it got carved into its own small sibling — learn (the soma puzzle-bin piece, now ready): a surface-agnostic gate(experiences, champion, train=…, score=…) any loop can use for the same never-ship-a-worse-prior guarantee. werbos’s gate delegates to it when installed, byte-identical fallback otherwise (the soft-dep pattern capture established), proven by a parity test. Five soma pieces are ready now; gate and search are the last two planned.

The answer bank learns to grow (c3b54c2). The un-bribable known-answer tier only helps for questions it has — but nothing model-derived can auto-enter, or it stops being un-bribable. So it grows the only honest way: a checkable question the loop can’t yet ground lands in an append-only review queue (propose); a human reviews the recurring ones and banks the TRUE answer (the model’s guess rides along only as a hint). A packaged read-only seed now layers under a runtime-writable user bank, so it grows on the operator’s box without mutating the shipped seed. Every hallucination caught becomes a question werbos can never be fooled on again.

Eight commits all told, ~24 new tests. The arc that opened with grade outside the model closes here: the model proposes; the policy learns only from what something outside the model confirmed — and the bank of what’s outside the model can finally grow.