The morning after the overnight LoRA train. v2-coder existed in two candidate forms (iter-1000, val 0.850 plateau; iter-2000, val 0.811 new low). We didn’t know which was actually better — and the TTT bench from yesterday told us both were worse than stock, which felt wrong.

Today we built the right benches, ran them, and learned the lesson that’s going to shape every future training run: val loss alone is a deceptive metric. Multi-axis benchmarking is the only honest signal.

What shipped

• werbos.games.code_bench — coding-skill bench over a fixed five-problem bank (fizzbuzz, reverse_string, two_sum, is_palindrome, merge_intervals), judged by pytest in isolated tempdirs. Reports problems-fully-passed and total-cases-passed. (668cee0)
• werbos.games.debate_bench — conversation-skill bench over five fun debate topics (hot-dog sandwich, pineapple pizza, cereal soup, tabs vs spaces, tomato fruit). Structural scoring on format_clean (no tool-marker leak), length_ok (80-2000 chars), non_refusal, and the key signal: held_position (on turn 2, rebutted instead of flipping). (included in 668cee0)
• werbos bench code and werbos bench debate CLIs — thin wrappers, sibling to werbos bench ttt. Reports land in ~/werbos/benchmarks/ so cross-model deltas are diffable. (included in 668cee0)
• The shootout — ran ttt + code + debate against three adapters (stock-3b, v2-coder@iter1000, v2-coder@iter2000) and measured 15 numbers. Triangulation kicked in.
• Production adapter promoted — iter-1000 is now the active ~/werbos/adapters/v2-coder/adapters.safetensors. Numbered checkpoints all kept in place for future A/B.

The shootout

iter-2000 lost two coding problems (is_palindrome, merge_intervals) that both stock and iter-1000 solved cleanly. Classic late-training overfit: the model memorized harder, train loss kept going down, val loss accidentally dipped on a lucky sample — but the real behavior on unseen tasks degraded.

Numbers worth keeping

AHAs

• Val loss alone is a deceptive metric. It told us iter-2000 was best (0.811 < 0.850). Real benches told us iter-1000 was best. Going forward: val-loss-during + multi-bench-after is the only honest training-quality signal.
• TTT was orthogonal to what v2-coder was trained for. The fine-tune shifted the model toward <tool_use> markers and away from clean JSON envelopes. TTT requires clean JSON. So TTT got worse on the adapter even though coding didn’t, and debate improved.
• Triangulation > single signal. With one bench we’d have thrown out the adapter. With three we kept it.
• Adapter v2-coder is a real improvement — just on the axes it was trained for. Held-position 80% → 100% on debate is a measurable, structural win.

The bonus skill that got built

/document-it — a Claude Code skill that reads git commits since the last documented SHA on the current branch, categorizes them by conventional-commit prefix, and updates registered doc targets (changelog, README, hugo log dir, roadmap). Per-branch state in <repo>/.document-it.json. Today’s entry — this very file — was the dogfood test.

Quote of the day

“Val loss alone is a deceptive metric. The right loop is val-loss-during + multi-bench-after.”

The line buried in the commit message that becomes the rule.

What’s next

• Pull the other-machine corpus, train a v3
• Try adapter stacking (v1 + v2-coder-iter1000)
• Wire the bench suite into a nightly cron (/schedule-able)
• The site git decision (still open)

Part II — afternoon

“Skills, personas, agents — locked in.”

After the morning shootout, the afternoon was design + first concrete artifacts for the agent system that doesn’t exist yet but now has a spec rigorous enough to build against. None of it landed in git (yet) — the skill lives at ~/.claude/skills/, the personas at ~/werbos/personas/, and the architecture is conversation.

What shipped (off-repo)

• /document-it skill at ~/.claude/skills/document-it/SKILL.md — the skill that wrote this entry by reading commits since the last documented SHA, matching the existing changelog voice, prepending entries, and saving per-branch state.
• ~/werbos/personas/jason-voice.md (109 lines) — the foundation persona for werbos when Jason is the user. Identity-level facts (M1, SR8, Pi 5, CryptoBlocks, parents alive, teal), tone calibration, what werbos does NOT do. Closes with the bar: “If your reply could have come from any cloud LLM, you’re failing.”
• ~/werbos/personas/coder-mode.md (100 lines) — the first role-overlay persona. Read-before-write, diagnose-before-retry, single-file-bias, no premature abstraction.

What got committed

• b82d6b6 — chore: gitignore .document-it.json (skill state file is per-machine local).

Architecture decisions locked in

The agent system design held five open questions yesterday. All five are now answered:

The composition rule: foundation persona is a config slot, not a hardcoded value. ~/werbos/config.toml has foundation_personas = ["jason-voice"] today; if the project ever ships to another user, they swap one line. Agent files never name the foundation directly — the runtime injects it.

AHAs (Part II)

• Foundation persona as a config slot is the trick. Solves “werbos is Jason’s” + “werbos can ship later” with one config line and no agent-file rewrites.
• /document-it has a real limitation: it sees commits, not off-repo artifacts. Today’s afternoon was ~75% off-repo (skill + personas + design). The skill should grow an --include-paths flag to track non-git surfaces. Not blocking.
• Persona blending is conceptually trivial (~15 lines: load files in order, concatenate with section headers). The hard work isn’t the runtime — it’s writing personas that actually capture voice without sliding into corporate-LLM mush.

Quote of the afternoon

“If your reply could have come from any cloud LLM, you’re failing. Werbos is his. Sound like it.”

The closing line of jason-voice.md. The thing every future agent inherits.

Part III — evening

“Bro made a webpage. With personality.”

The day werbos got a face, a real coding workflow, and stopped being just a CLI tool.

1. The first real-work artifact

werbos code (running stock 8B Llama via Ollama after the mlx OOM) was given a series of small prompts: ls, pwd, create boilerplate html, make bg green, add h1, fix duplicate tags. He shipped a working jason.html — green page, white “Hello World” — despite emitting half his tool calls as literal text instead of invoking them, plus duplicating <html> / <head> tags.

Diagnosis: stock 8B is what v1 trained out, what v2-coder fixes more, and what v3 (with the other-machine corpus) should mostly close.

The user’s takeaway:

“I can’t say ‘make me the final product’… but maybe I can get it to do basic things if I guide via small tasks.”

That’s the entire current ladder. Decomposition + correction is the workflow.

2. Tauri 2 desktop scaffold (3949c5d)

werbos/ui/src-tauri/ lands. Mirrors the existing CryptoBlocks / EnvSync / swagblocks pattern: Tauri 2.11 inside the React project, single config that targets desktop AND iOS AND Android. Identifier dev.werbos.app, werbos_lib crate with mobile_entry_point macro ready so tauri android init can drop straight in.

New npm scripts: tauri:dev, tauri:build, tauri:android:init, tauri:android:dev, tauri:android:build. cargo check passes in 35s. Mobile target init is one rustup migration + NDK setup away.

3. Squirrel splash + shell-in-chat + whimsical loading (77b9cf5)

Three things in one chat-session pass:

• Splash banner at werbos/cli/_splash.py — pixel-art squirrel mascot in three forest/sage greens, printed once at the top of werbos chat. Skipped when stdout isn’t a TTY.
• Shell tool registered in chat by default. Bare werbos (drops into chat) now has the run_command shell tool with a confirm- before-execute leash. Flags: --no-shell, --yolo, --workdir.
• Whimsical loading messages at werbos/wait.py. 59 words across 5 vibe groups. The chat REPL prints a rotating + Razzmatazzing... (12s · thinking more) line in werbos sage green during the AI turn. Tail flavor escalates — “thinking” → “still going” → “this one’s chewy” → “honestly might be looped” → “send help (or just Ctrl-C)”. Mirrored to TS at werbos/ui/src/lib/wait.ts; React typing bubble uses the same line.

4. chat ↔ code parity (aa04574)

Mid-changelog the user pointed out: “werbos code and werbos chat would work 1:1 the same via the cli” — and they should. Pulled the splash + status loop into werbos/cli/_status.py. Both interactive REPLs now show the squirrel splash on launch and the rotating sage- green loading line during AI turns. Single-shot werbos code -p <prompt> skips both so scripted output stays clean.

Numbers worth keeping (Part III)

AHAs (Part III)

• “You drive, werbos types” is the workflow at this scale. Not a werbos bug; it’s the whole agent-loop genre at small-model scale.
• Werbos has a face now. Squirrel mascot + sage palette + the rotating loading words give the CLI a personality nothing else has.
• Tauri 2 is one unified config away from mobile. All the scary stuff (Rust mobile targets, NDK, Android Studio project) is opt-in via flags.
• Color discipline matters more than expected. Matching the loading line to the splash green made the CLI feel cohesive.

Quote of the evening

“i ecpect a year+ of training to get this closer… but this is cool… it created something…”

The right frame. It created something is the meaningful word.