Base training · measured · complete

The base, by the telemetry

werbos's owned 730M base — 10.68B tokens, one rented H200 — trained to a held-out loss of 2.049 — read seven ways. Then a cautionary eighth: the first fine-tune, where a beautiful loss curve lied. Every phase and rate below is computed from the run, not eyeballed off a curve.

loading…

1 · Loss vs step

raw + smoothed to step 146.7k, then the final checkpoint · best · held-out

2 · Perplexity vs step

log axis · PPL = e^loss · the early collapse

3 · Learning bought per 1,000 steps

Δloss / 1k — where the learning actually happens (note the plateau thrash)

4 · Learning rate vs loss

the relationship, post-warmup — loss as the cosine LR decays 100%→10%

5 · Tokens vs loss

the cross-run axis — how far 1B, 5B, 10B tokens gets you

6 · Efficiency

loss reduction per billion tokens, by phase

7 · Phase breakdown — boundaries derived from the telemetry

phase	steps	Δloss	perplexity	Δloss / 1k	learning rate

8 · The conversation tune — a beautiful curve that lied — a different run

336M talk model · fine-tuned on 1M card-grounded conversations · loss 5.23 → 2.53, held-out 2.59, no overfit — and the model got worse