B.18 — coverage wasn't the problem: valence hits a capacity wall

Sun, Aug 2 2026

The talk model answers 'I lost my job today' with 'That's awesome!' — a valence flip. B.17's persona corpus had empathy at ~3% of responses, so the first theory was a coverage hole. This tests that theory to destruction: a 5M model retrained with empathy upweighted to 16.4% (25k freshly-minted balanced pairs), identical size and 16,880 steps. It still flips. At 3% it flips; at 16.4% it flips. That rules out coverage and points at capacity: the model learns the phrases of empathy but not when to use them. Held-out loss came in lowest of every arm while behaving no better.

T.M.B. #18 — the fix that proved the fix was impossible. The valence flip (negative input → cheerful reply) looked like a data-coverage bug. So we made the coverage 5× stronger and trained again. It didn’t help — which is a cleaner, more useful result than a fix would have been: it moves the problem out of the training set and into the weights, where the runtime spine already handles it.

The setup

The owned 5M talk model (B.16, B.17) answers emotional openers with cheerful filler — “I lost my job today” → “That’s awesome!”. B.17’s persona corpus was emotionally flat: measured over 273k ours pairs, empathetic responses to a negative input ran ~3% while cheerful ran higher. Hypothesis: the model can’t learn a 3% pattern swamped by the majority key, so fill the coverage hole.

We retargeted the scenario minter to produce three registers — emotional (disclosure → comfort, grounded in the therapy libraries), practical (domain help), and light — and ran a 4-hour mint: 25,404 balanced pairs, 12,340 of them empathetic. Then a training mix designed so empathy is a learnable fraction, not a rounding error:

bake_valence.jsonl = ours[:100,000]  +  balanced × 2  =  150,808 pairs
   empathy 16.4%   |   practical 8.0%   |   light 75.6%

Two 5M byte-level models, identical architecture (n_embd 256, n_layer 6, ~4.94M params) and identical 16,880 steps — the only difference is the data:

bake_a — ours only (273k, the B.17 baseline)
bake_v — the valence mix above (empathy upweighted to 16.4%)

The number

model      corpus                 held-out loss   clean   input-sensitive
bake_a     ours (empathy ~3%)         1.087        0.92        0.92
bake_v     valence (empathy 16.4%)    1.052        0.94        0.90

bake_v has the lowest held-out loss of any talk arm to date (below B.17’s ours 1.087 and blend 1.190) and passes every talk_bench floor. On paper it’s the best model. It is not.

What each actually says

The talk_bench floors can’t see valence-appropriateness (its own docstring: “responsive is variation, not relevance”). So the real test is a switch probe — negative inputs should get empathy, positive inputs should stay light — decoded with the shipping sampler (top-k 3, temp 0.5):

"i lost my job today..."          bake_v → "that's awesome! have you tried any special sparks yet?"
"barely left the couch, drained"  bake_v → "that's amazing, what was the worst one thing..."
"just booked a trip to portugal!" bake_v → "Aw, that sounds like a perfect excuse to be the day."

Still “that’s awesome” to job loss. The greedy training-log samples showed the opposite collapse — empathy phrases on a friendly greeting (“Nice to see you” → “That sounds really heavy”) — which means the model owns both openers and simply cannot condition on the input’s valence to pick one. It learned the vocabulary of empathy; it did not learn the moment.

What actually moved the number

Nothing that mattered. Empathy coverage went from ~3% to 16.4% — a 5× stronger signal, freshly minted, grounded, balanced — and the switch behavior did not improve. That rules out the coverage hypothesis. The remaining explanation is capacity: a 5M byte-level model can render register-appropriate phrases but can’t read the emotional content well enough to select the register. Same family as the reader wall (a 1M numpy reader once beat a 57M pytorch one): comprehension does not survive compression into these weights. And the loss ranked the least-useful model first again — the fifth confirmation that held-out loss ranks talk models backwards, because predictable warm filler is exactly what a cross-entropy metric rewards.

The honest caveats

One recipe, not a sweep. We tested 16.4% at one size, one LR, one step count. A different mix or a curriculum might move it; the strong prior (reader wall + the B.16/B.17 line + this result) is that it won’t, but this is one point, not a curve.
The switch probes are hand-written. Six openers in one operator’s phrasing — a floor, not a ceiling. They show the flip persists; they don’t quantify how often.
16.4% may also over-correct. The greedy samples hint at empathy-on-everything at higher blend — the mirror failure — which is its own argument against solving valence in the weights.
Every number here re-runs: mint_scenarios.py --heavy-frac 0.45 --practical-frac 0.25, the mix build (ours[:100k] + balanced×2), train_talk_standalone.py --corpus bake_valence.jsonl --sizes 5 --steps 16880, and talk_bench.py bake_a bake_v plus the switch probes.

The verdict

Coverage was not the problem, so more data is not the fix. Valence-reading belongs in the comprehension layer — interpret reads the emotion, plan declares the empathy need, gather retrieves the response, and the weights only speak it — all of which is deterministic, owned, and was built the same day (compose/interpret.py, plan.py, spine.py). The 25k empathy corpus is not wasted: it is now SPEAK-training (the model can render warmth) and the library the empathy lane gathers from. The training run’s real output wasn’t a better model — it was the line between what a 5M cursor can hold and what a library has to.