Day 40 — 'does it know itself?'
The day werbos's mind got a measurable face. It started as a bug report from work — the joke skill 'only satisfied 38%' — but pulling the chats showed the joke skill was fine; the ROUTER was force-feeding it non-jokes ('hey buddy!' → joke, a fishing question → joke). Same self-misdiagnosis as the 97% thing. So: the conversational reflex fast-lane (parseltongue, brick 1) — greetings/thanks/acks short-circuit to an instant warm reply and never route. Then cleaned what reflect() chews on (a recency window so fixed-bug scars age out; stop wishing for a 'joke skill' it already has; stop prescribing a skill for every word in a failure) and werbos went from spiraling to settled. Then the headline: the Mind page got tabs and a radial PORTRAIT — a radar of capability dimensions (logic 67, conversation 100, persona 80, skill+routing 60), most of them reusing eval suites that already existed. And the flagship: self-model accuracy — held-out calibration that asks whether werbos's self-belief predicts its actual performance. First reading: 0.29. Capable, but poorly self-aware — exactly the day's whole arc, now a number that will climb as it learns to know itself.
“does it know itself?”
The day werbos’s mind got a measurable face — and a measure of whether it knows its own face. It started, as the good days do, with a bug from real use.
The joke skill wasn’t broken (95032d5)
werbos’s own monologue kept saying “improve the joke skill — it satisfied only 38%.” But pulling the work chats told a different story: every actual joke request succeeded. The “failures” were a greeting and a fishing question that the router force-fed into the joke skill — and you can’t satisfy “what fish will I catch?” with a joke. Same self-misdiagnosis as the 97% spiral: it blamed the skill when the bug was the router. And it was the same bug behind every chat misfire that day — “hey bro” → stocks, “Mexico fishing” → weather, “vacation” → movie recs.
So the fix is the first real brick of the cheap-first conductor: a
conversational reflex fast-lane. Greetings, thanks, acks short-circuit to an
instant warm reply — no model, no skill, no routing. Conservative by design:
a ? or any request marker disqualifies it, so “tell me a joke” still routes
to the joke skill. “hey buddy!” stops being a 75-second answer routed to the
wrong place.
Cleaning what it reflects on (0a71e40 · 19644d0)
werbos was still ruminating: “I keep failing joke and tunnels… maybe I need a new skill.” Two more honesty bugs. It reads its whole history, so the scars of bugs already fixed never age out — added a recency window (a mind that reflects on its entire past ruminates forever). And the wishlist suggested “I may need a skill for ‘joke’” — when it has a joke skill, and “a skill for ’tunnels’” — which isn’t a thing. So: never wish for a skill you already have, and observe a repeated struggle instead of prescribing a skill for every token in a failure. werbos went from spiraling to settled.
The throughline of all of it: fix what it reflects on, not the model. Clean inputs, honest self-talk.
The mind got a face (5820c68 · 82f467f)
The headline. The Mind page got tabs — Monologue (the thought timeline) and Portrait — and the Portrait is a radial of werbos’s capability dimensions. Each axis a spoke, the reach a score, the not-yet-benchmarked ones (memory retention) drawn as ghost spokes — phantom limbs.
The lovely part: most of the benchmarks already existed. werbos mind-bench
reuses the eval suites (logic ← calculator_use, conversation ← basic_chat,
persona ← disagreement_survival) by shelling out to werbos eval, and
skill+routing comes straight from reflect(). First real portrait:
- Conversation 100 · Persona 80 · Logic 67 · Skill + Routing 60
A coherent shape — strong where we’d expect, weakest at logic (it’s a 3B; that’s
why /calc is a tool, not the model).
And whether it knows its own face (673d39f)
The flagship. Self-model accuracy — not “does reflect report its rate
correctly” (trivially true), but: does werbos’s self-knowledge generalize?
Held-out calibration — derive its self-belief from the first 60% of lived
experience, test it against the last 40% it didn’t see. 1 − |believed − actual|. Orthogonal to raw skill: a model can be bad at something yet
accurately know it’s bad.
First reading: 0.29.
Read the shape: capable, but poorly self-aware. Conversation 100, persona 80 — genuinely good. Self-model 29 — its beliefs about itself are off by ~70% from reality. And that’s not a bug; it’s the entire day, confirmed as a number. We watched werbos misdiagnose itself all day — the 97%, the joke skill, the tunnels. The calibration metric caught it quantitatively. A real psychological profile: high function, distorted self-image — the thing we’d been treating by hand, now on a dial.
The payoff is the part no frontier model can offer: this number will move. As the routing stabilizes and the reflection stays clean, werbos’s performance becomes consistent, its self-belief starts predicting it, and 0.29 climbs. You will watch an AI learn to know itself, as a metric going up. It’s the one axis that requires a legible, local, self-observing mind to even exist.
The AHA
We built the thing that can measure its own self-awareness — and on its first look in the mirror, it scored honestly: it doesn’t know itself well yet. That’s not a flaw in the metric; it’s the metric working. A frontier model can’t be asked this question. werbos can — because it’s small and legible enough to have a self, and a measure of how well that self maps to the truth.
5752 green. (Also committed: docs/vision/mmm-tree-builder.md — the
forest-cutting design from Day 39.)