The industry ships a press release that says the model is too dangerous to release, and the stock goes up. We did the opposite. We put the whole training run on the wall.
Our owned base model — 730M parameters, 10.68B tokens — is training right now on
a single rented H200. Instead of a hero number and a scary name, we published the
telemetry: seven panels at /base-curve/, every one derived
from the run itself.
What the receipts say
- Perplexity fell from ~20,000 to ~10. That’s the model going from guessing blind across the whole vocabulary to choosing between roughly ten words for each next word. That’s learning to speak, plotted step by step.
- $18 per loss-point. Not “$5 million.” Eighteen dollars. The whole run comes in around $150 — a fluent, owned base model for the price of a nice dinner.
- 0.79 loss per billion tokens. The efficiency, measured, so the next run can be compared on the same axis instead of on vibes.
The point of the price isn’t “look how cheap.” It’s that the barrier moved. AI from the ground up is no longer a capital problem — it’s a knowledge problem. You don’t need a data center. You need to know which moves, in what order.
The graph that proved us wrong
Here’s the part we’re proudest of. One panel plots learning bought per 1,000 steps — the derivative of the loss. We’d been telling a tidy story about a “second wind” during the final annealing phase, where the model re-accelerates.
The data said no. Whole 1,000-step windows in the middle of the run go negative — the loss actually rises — the model thrashing sideways on a plateau. The annealing phase doesn’t re-accelerate at all. What it does is quieter and better: it turns that thrash into a steady, monotonic descent that keeps banking new lows.
We shipped the graph that corrected our own marketing. That’s the whole ethos in one panel: measure it, and when the measurement disagrees with the story, the measurement wins.
The graph we refused to draw
There’s an obvious eighth panel: loss versus tokens across every model size we’ve trained — 30M, 85M, 350M, 730M — the classic scaling curve. We didn’t draw it, and we put a note on the page explaining why.
Those models trained on different corpora. Loss is only comparable within one corpus; stacking them on a single axis would be ranking a swimmer against a cyclist by their heart rate. It’s the exact “different rulers” mistake we refuse to make — so instead of a pretty, misleading chart, there’s an honest paragraph. Refusing to draw the dishonest graph is more us than drawing the beautiful one.
Go look
The base is still climbing as you read this — best loss so far 2.15, a few hours from the finish. You can watch the real numbers move at werbos.netlify.app/base-curve.
Trust-me-bro, except with receipts. That’s the only kind of benchmark we post.