B.17 — the hive mind scores best, and that's the warning

T.M.B. #17 — the metric prefers the model that stopped listening. B.13/B.15/B.16 found that held-out loss ranks talk models backwards. This is that finding taken to its limit: a three-way bake-off where the model with the best loss is the one that dissolved into a generic voice with no individual behind it. Same architecture, same 16,880 steps; only the data differs.

The setup

Three 5M byte-level models (n_embd 256, n_layer 6, ~4.94M params), each trained 16,880 steps on a different corpus, then run on the same eight novel openers neither had trained on:

a. ours     273k pairs   (our owned conversation corpus)      -- starved for 5M (optimal ~1.4M)
b. blend    962k pairs   (ours + ~689k SODA, converted)       -- fed
c. SODA     962k pairs   (pure SODA, converted dialogues)     -- fed

SODA (allenai/soda, CC-BY-4.0) is a 1.5M-conversation dataset itself distilled from a large model. Its multi-turn dialogues were flattened to input->response pairs.

The number

What each actually says

a. OURS (starved)     "my flight got delayed"  ->  "That's awesome! What do you think..."
   coherent but TONE-DEAF -- mode-collapsed to a safe phrase. Trapped in a small distribution.

b. BLEND              "my flight got delayed"  ->  "I know that feeling. Maybe a short something soon?"
   READS the complaint. Empathetic, has range. Tails get messy. The interesting middle.

c. SODA ONLY          "just adopted a kitten"  ->  "I know, but I can't wait to see it. It's always
                                                    been a long time for you."
   fluent, warm, and UNTETHERED. It doesn't answer the kitten. It generates generic emotional
   dialogue that could attach to anything. The hive mind.

What actually moved the number

Read c again. It is grammatically smooth, emotionally toned, and completely disconnected from what was said. It learned SODA’s shape – two characters emoting at each other – and lost the person in front of it. And that is exactly why its loss is lowest: generic emotional dialogue is the most predictable text you can produce, so a model that always generates it hedges cross-entropy better than one that risks a specific, input-shaped reply.

loss says:      c > a > b     (SODA best)
quality says:   b > a > c     (blend best, SODA worst)

The metric and the quality are inverted, and this run shows why it is dangerous rather than merely curious: optimizing held-out loss alone drives a talk model toward the hive mind – the fluent, self-less voice that scores well precisely because it stopped taking the risk of listening.

The honest caveats

• The held-out splits differ per corpus, so the loss numbers are not a clean head-to-head; the direction (SODA lowest, blend highest, quality inverted) is the finding, not the exact deltas.
• Eight openers is a read, not a metric. A distinct-reply-rate over hundreds would quantify what these eight illustrate; the gap is large enough to act on, not yet large enough to publish as a score.
• All three degrade in the tails – 5M byte-level, the B.9 coherence ceiling has not moved. What differs is whether the model reads you, not whether it stays coherent for a paragraph.
• SODA is CC-BY and usable, but this is one dataset; the finding is about pure-collective training, not about SODA specifically.

The verdict

Ship the blend, capped, with the owned pairs dominant – it is the only one that both reads the input and has range. Never ship pure SODA: it is loss-optimal and self-less. And treat the metric with suspicion for open chat – a number that rewards the model for not listening is a number pointed the wrong way.

The deeper result is the one that matters beyond this project: a collective corpus, absorbed whole, produces a voice with no individual behind it – and the naive objective prefers it. Preserving the individual is not the default; it is the thing you have to build against the gradient.