Day 83 — the weights can only speak

“the weights can only speak.”

Some days you fix the bug. Some days you prove the bug can’t be fixed the way you thought — and that turns out to be the more valuable thing.

The model that said the wrong thing

The bake-off left a model that answered “I lost my job today” with “That’s awesome!” A cheerful reply to grief. The first theory was coverage: the training corpus was almost all upbeat small-talk, empathy was a 3% rounding error, so the model learned one key and played it for everything.

So we filled the hole. The scenario minter got a second register — heavy emotional arcs where one person discloses something hard and the other comforts, with the listener’s stance drawn from the graded therapy libraries (validate first, don’t minimize, never say “that’s awesome” to pain). Then a third register — practical help, recipes and fishing and fitness, grounded in their libraries — so the model would have to learn to switch keys, not just add a default. The persona roster grew to 64, weighted and ordinary both: a chef who wants recipe ideas, a fisher, a developer with no current crisis. A four-hour mint produced 25,000 balanced conversations, half of them empathetic.

The model belongs last

Somewhere in the middle, the architecture caught up to what the pieces were becoming. The insight is small and it reorganizes everything: the model is the last step, not the first. A message shouldn’t go straight into the weights. It should be understood, then planned, then the evidence gathered — and only then does the model turn that evidence into language.

So we built the missing box. interpret — the detective — now reads the emotion in a message. plan takes that reading and declares the needs: “I’m sad” needs empathy; “how do I grill salmon” needs a recipe; “I’m lonely” needs empathy and memory. Retrieval stops being “search everything and hope” and becomes “resolve exactly this list.” Each stage can be scored on its own. The weights’ whole job shrinks to the last stage — say it well.

Style is a library, not a talent

Then the day turned playful, and the playful thing turned out to prove the serious point. “Convert Genesis to pirate” is style transfer of text we already own — the tractable cousin of writing like Dickens, because the content is fixed and the result is checkable. A style “pack” is a small owned library: a lexicon, some syntax tics, interjections, and its own detector. The engine applies it and two graders check the output — does it read as the style, and does it still mean the source?

It worked, and then it scaled: the entire 31,095-verse Bible transformed in four seconds, every verse graded for meaning, with the grader honestly flagging the handful where “the Lord” became “the the Cap’n” — and then flagging the regression the fix introduced when God and Lord collapsed to one token. Prove, don’t trust, working on itself. Five more packs followed — lolcat, Shakespeare, cockney, gen-z, corporate — each about fifteen lines of data, the engine never touched. And a blend dial: zero pirate to full pirate, verified to raise the style score while pinning the meaning. The whole thing wired into the runtime spine, so “convert Genesis 1:1-3 to pirate” now flows through all four stages, the source pulled verbatim from the owned concordance and never recited from memory.

The lesson, paid for in a training run

The empathy corpus deserved a real test, so we ran the coverage theory to destruction: retrained a 5M model with empathy upweighted to 16.4% — five times the signal that failed. It still called job-loss “awesome.” At three percent it flipped; at sixteen percent it flips. That isn’t a coverage problem — it’s a capacity one. The model learned the phrases of empathy (“that sounds heavy,” “it’s tough”) but not when to reach for them. Reading the emotion in front of you is comprehension, and comprehension doesn’t fit in these weights. The held-out loss came in lowest of every arm while the behavior was no better — the metric, once again, ranking backwards.

Which is the day’s whole point, arriving from two directions at once. The morning’s spine wasn’t a nice- to-have sitting beside the model. It’s the mechanism. The weights can only speak. The knowing — what the person means, which key to play — has to live in the part of the system that can actually read, and that part is owned, deterministic, and already built. The 25,000 empathy conversations weren’t wasted; they changed jobs. They’re what teaches the model to render warmth once something upstream has decided warmth is what’s called for.

The evening — the spine grows lanes

The frame built in the morning was empty: understand, plan, gather, speak. The evening filled it with organs — one capability per lane, each honest about what it is.

Memory, as a place instead of a window. Tell werbos your favorite color is bitcoin-orange, talk for fifty turns about anything else, then ask — and it answers. Not because the fact rode along in a context window that grows and forgets, but because PLAN declared a memory need and GATHER read it off disk. Constant memory, not context: state a fact once, recall it whenever, at O(1) (c0423cb).

A gate that would rather abstain than bluff. “How do I create an angular component” used to get answered out of the physics library — the router grabbed the biggest adjacent shelf and confabulated. The fix isn’t a bigger router; it’s a domain-consistency gate: if the question’s domain and the retrieved card’s domain disagree, don’t answer — abstain (b0498a5). To make that judgment sharp, the coding knowledge got a modular taxonomy — languages, frameworks, general terms, a strong/weak signal split (61a9833) — and the weak-signal words like service, directive, module closed the last gap, where “angular service” (one thin cue) still slipped through to a bluff (bcaa525). This is the Wednesday coding weigh-in prep in miniature: a model that abstains cleanly on Python and Angular is worth more than one that fakes them.

A borrow lane, behind one seam. Some things werbos doesn’t own yet — summarizing a web page, brainstorming a list. The rule stands: rent that, but say so. The borrow tier does exactly the two rent-allowed shapes and labels every reply as rented (e5cdf3b); offline it falls back to an owned, extractive summary that lifts real sentences verbatim, so it physically cannot hallucinate (476b82e). The generator sits behind a single constant — the day the owned renderer is good enough, one line changes and the lane becomes ours.

And the playful lanes got sharper. The speak-pack transform wired all the way into brain_reply (a2e5b8b), and gen-z grew from word-swaps to real phrase rewrites — “came to him saying” becomes “pulled up to him and were like” (e82a492), which quietly unlocks gen-z prompting (write me a for-loop in gen-z) for free. Tarot learned that “relationships” is plural and deserves the five-card spread (dd5ba54). And a new on-ramp: point recipe_ingest at a recipe URL and it reads the page’s own structured data into cited werbos cards — extracted, not generated, every card carrying its source (1fbd284).

Seventeen days to the gauntlet. werbos ended the day knowing something it didn’t know in the morning: exactly where the line runs between what a small model can hold and what a library has to. That line is the architecture.