Day 33 — 'public-prep and the latency hunt.'

“public-prep and the latency hunt.”

The repo is supposed to go public. That means two things had to happen: it has to be clean enough for a stranger to read, and it has to be fast enough that the first thing a stranger types doesn’t time out. Day 33 did both — and the second one turned into a four-bug archaeology dig.

Public-prep (3eb23d1, e0952f4, af2a5d6)

• Scrub — /Users/jasonsutter → /Users/operator, jason@<tailnet-ip> / the homelab IP → operator@homelab, SR8 → homelab across docs, comments, and the security-review tree. deploy/sr8/ → deploy/homelab/. The public GitHub handle stayed (it’s already public). Comment/docstring-only in code.
• Test repair — 8 stale tests fixed, including test_ollama_chat_ssrf which still pinned the original over-strict Bo-001 behavior; rewrote them to pin the documented local-first-aware policy. Found and closed a real gap while there: the ollama host guard wasn’t rejecting multicast on either the literal or DNS path. 5,438 green.
• Sprint hygiene — DATA.1 / GH.1 / GH.2 / SE.6.1 docs moved to shipped/; README + ROADMAP refreshed.

nightly_train.sh (9cbe351)

The train-daemon orchestration existed; now it has a real launcher. Operator-generic nightly script: rsync curriculum outcomes + docker-cat skill-smoke samples from the inference box (env-configured, no hardcoded hosts), then werbos train-daemon run with an MLX trainer default mirroring the werbos-0.0.1 recipe (3B, 500 iters). Scheduled via launchd on the training box. The ratchet now clicks while the operator sleeps.

DS.1 Day 1 — theme-pack resolvers (dd9aa05)

Port of the operator’s own Lectern typography + button builder (provenance: operator-authored, verified in Lectern git history) into werbos/design/: frozen-dataclass theme schema, modular-scale type resolver, variant/state button inheritance, and — the load-bearing security piece — safe_css with a char allow-list + dangerous-function deny-list (including the /**/-comment gap between a name and its paren), re-applied at BOTH the paint seam and JSON load.

The SE.10 latency hunt

A bare “hey buddy” on the CPU-only homelab box took 176 seconds. The model itself answers in 1.4s. Everything between was overhead — four separate bugs, each one only visible after fixing the one in front of it.

1. keepwarm evicted its own model (e3e1b2c) — Ollama treats the same model at a different num_ctx as a different instance. Chat asked for num_ctx=8192; keepwarm pinged with no options → default ctx → every 60s ping evicted the chat instance, and vice versa. Every turn paid a full reload. KeepWarmDaemon now pins num_ctx.

2. the persona was 12.4K chars (7b6bb24) — the persona layers alone are ~3.1K tokens, and prefill on that box is 58 tok/s — 53s before the model can speak, every turn. Shipped a hand-condensed compact persona (~85% smaller, every load-bearing rule kept and test-pinned) behind werbos serve --compact-persona, plus stripping HTML comment headers (default.md’s alone was ~480 dead tokens/turn).

3. the AO.0 bug, reborn (d552da1) — the dispatch_skill tool description inlined the full 125-skill catalog (name + description, ~3.1K tokens) into every turn’s tool schema. ~55s of prefill before “hi”. Names-only catalog: 12,461 → 2,368 chars. Direct-probe evidence: identical payload minus the tool docs answers in 5.1s.

4. you can’t fix what you can’t see (7ee009f) — shipped werbos latency first, so every chat POST records one chat/turn trace row with total wall-clock + route. The number every SE.x speed sprint moves now has a meter.

And 26 new precook seeds (858149f) — precookable skills grew 3 → 30 (reference / wellness / finance explainers; creative and time-dependent skills explicitly stay live). On the CPU box a cache hit is <50ms vs ~30s live. Precook coverage is the speed strategy there.

Numbers worth keeping

• “hey buddy”: 176s → ~30s
• Prefill math @58 tok/s: persona 52s → ~9s; catalog 55s → ~5s
• Precookable skills: 3 → 30 (46 seed corpora)
• Tests: 5,438 green after the repair pass

AHAs

• “Local models don’t suck. Using them like frontier models is what sucks.” Every one of the four bugs was framework overhead, not the model. The bare model was always fast.
• The AO.0 bug came back through a channel nobody was watching — the tool schema. Same disease (inline docs blowing up prefill), new vector. Fixing a bug class once doesn’t immunize the other doors.
• Ship the meter before the fix. werbos latency turned “feels slow” into four named, separately-attributable numbers.

Tomorrow

The box is fast enough to demo. But it’s still answering some prompts with raw JSON — the skill router hallucinates tool calls as text. That’s the next thing to chase.