Day 6 — 'The infrastructure is the artifact.'
Thirty-five commits, 909 → 1266 tests, six sprints closed in one day (B.1/B.2/B.3 TTT+C4 end-to-end, S.1 security hardening via Black-Team→Red-Team agent loop, A.1 foreground-busy adapter, A.2 background work queue + executor, K.1 per-repo .werbos.md context, U.1 mid-stream JSON swallower + CLI streaming parity, T.1 werbos doctor). One LoRA trained (v1-c4) at 3% as O — corpus design lessons captured. Plus a five-game suite expansion (blackjack, chess, battleship, checkers, heads-up Texas hold'em) that turns the entire game surface into parallel dogfood streams for the reasoning + logic ladder. The day's actual artifact is the platform, not the LoRA: corpus generator, bench harness, heuristic fallbacks, hardened HTTP surface, observe adapter, task queue + executor, project-context discovery, streaming polish on every surface, and a games surface where each new game = a new corpus path.
“i like winning… but i like better training data.”
That sentence — said mid-session, watching v1-c4 lose its third straight game to the heuristic baseline — captures the day’s load-bearing lesson. The bench eventually came back at 3% as O versus the heuristic, with 0 forfeits across 100 games. The format trained perfectly: every reply was parseable JSON with a legal column. The strategy did not. v1-c4 played at random-policy level.
And yet today’s win count is sixteen commits and 171 new tests,
because the LoRA is one piece. Around it, a lot shipped. The
corpus generator works (10K rows in seconds). The bench harness
works (heuristic crushed v1-c4 cleanly, exactly as expected for a
weak LoRA). The Connect4 React UI is playable end-to-end with a
heuristic fallback that doesn’t need any model at all. The
security audit found one critical and four highs and Red-Team
fixed seventeen of them. The two god-function handler factories
got flattened. And werbos.observe.make_foreground_busy_callable
plus GET /api/observe/foreground-busy shipped — werbos’s first
public adapter against the observe sibling’s spec. Tomorrow’s
agents have somewhere to land.
Morning — TTT closes the dogfood loop
Three sprints close out the per-move-labels-to-Rate-UI arc that started in design conversations late on Day 5.
Per-move labels for TTT (
d808739). The React GameView now renders a<MoveList>below the board with one<MoveRow>per played move: 👍 / 👎 + an inline note field. Reuses the existing/api/runs/{kind}/{id}/messages/{msg_id}/ labelendpoint by extending it forkind="tictactoe". The label sidecar lands on disk as<game-id>.label.jsoneven before the game has an archive —LabelStorealready supported that — which dropped the sprint estimate from 3 hrs to 90 min. Caught a side aphorism worth keeping: “if the AI could think about 2 options, the best move for it AND given the opponent’s POV what they’re about to do, then weigh both.” That’s the two-POV-reasoning idea. Captured as BACKLOG 5.1.5.TTT React-flow PFM archives (
e39d3a4).werbos/server/ttt_archive.pyconverts a React-supplied game state (theGameStateshape the UI already maintains) into the same(SessionMeta, chain, tool_calls)PFM triple a CLI-played game produces. Idempotent overwrite per move withoutcome="in_progress"for mid-game snapshots, so a crash mid-game still leaves a valid partial archive. TheSessionMeta.adapter_pathfield gets populated for v5/v6 corpus attribution — labeled-blunder mining from this corpus knows which model played each move.TTT runs in the Rate UI (
491ecba).load_archivedispatches bymeta.game_type; the TTT branch builds chat-shaped messages fromKIND_NUDGEevents, with stablem-Nids. Position names like"X · top-left"match what the GameView showed during play, so labels round-trip. The Rate UI’s kind-filter dropdown adds “TTT”. Now: play a game → archive lands → Rate UI sees it → labels apply. Full dogfood loop closed.
Late morning — Connect4 in three slices
User chose Connect4 as the next game per the locked v5+ roadmap (TTT → Connect4 → blackjack → battleship → checkers → poker → Risk; chess is companion-mode, not training).
B.3a — primitives + bench (a936869)
werbos/tools/games/connect4.py shipped with the full board
shape: 6×7, gravity-based apply_move, all four win directions
in one cell-walking scan, a heuristic opponent that goes
win > block > center-weighted random. The match harness in
werbos/games/connect4_bench.py plays N policy-vs-policy games
with seed-reproducible RNG and forfeit accounting — a buggy
policy that returns an illegal column doesn’t crash the bench;
the offending side just loses cleanly.
No oracle: Connect4 is solved (Allis 1988 proved first-player wins) but a search-strong-enough oracle is too expensive for per-iteration bench. Score by match outcome only. 36 new pytests pinned every direction of the win-scan, the gravity rules, and the heuristic priority order.
B.3b — playable React UI + archives (252549b)
Connect4View.tsx renders the 6×7 board as discs with
hover-to-preview gravity (so you can see where your piece would
land before clicking) and win-line highlight when the game ends.
The per-move rating row from B.1 reuses cleanly — labels are
game-agnostic, only the position-rendering string changes
(“X · top-left” for TTT, “X · col 3” for C4).
The two new server endpoints /api/games/c4/move and
/api/games/c4/archive follow the same shape as TTT’s. The big
twist: when no WERBOS_C4_ADAPTER env var is set, c4_move
short-circuits to the pure-Python heuristic_policy. The UI
is playable from day one without any LoRA. That’s deliberate —
it lets Jason generate labeled-blunder corpus now, against a
non-trivial opponent, before v1-c4 ever exists.
The architectural side-payoff: extracted
werbos/server/game_archive.py from the TTT-specific
ttt_archive.py. Each future game (blackjack, battleship, …)
becomes a small descriptor (apply_move, nudge_payload,
tool_call_extras, initial_board) instead of a 250-line copy.
The Archaeologist later called this out as the kind of refactor
that ages well — “will survive.”
B.3c — corpus + bench scripts queued (d91067d)
scripts/build_c4_corpus.py samples ~5K unique O-to-move
positions via mixed self-play (50% heuristic-vs-random, 25%
random-vs-heuristic, 25% h-vs-h). Each labeled by a priority
oracle: immediate-win > immediate-block > create-threat >
depth-3 alpha-beta minimax with a center-weighted leaf eval.
Mirror augmentation (L-R only — the board’s not square so no
rotation symmetry).
scripts/bench_v1_c4.py is the post-training one-liner:
heuristic plays X (the advantaged side per Connect4 solving),
v1-c4 plays O (disadvantaged), 100 games, target >55% as O.
Midday — v1-c4 trains, then fails clean
Generated 10,016 rows (user chose 10K over the 5K option).
Distribution: 304 immediate-wins / 869 blocks / 1,456
create-threats / 2,379 depth-3-search rows / × 2 mirrors.
Training run: 2000 iters, batch size 1, learning rate 1e-4 on
mlx-community/Llama-3.2-3B-Instruct-4bit, ~30 minutes on the
M1, peak 3.4 GB. The loss curves looked phenomenal:
- Iter 1: val loss 3.710 (baseline)
- Iter 60: train loss 0.118
- Iter 100: val loss 0.094
- Iter 200: val loss 0.092
Train and val tracked together. Looked like a clean run.
The bench told a different story:
v1-c4 (O) won 3/100 = 3.0% vs heuristic (X) 97 wins, 0 draws
avg plies/game: 16.63
total time: 2703s
B.3c target: v1-c4 wins >55% as O. Result: BELOW TARGET
3%. The random-policy baseline is 0-3% as O against the
heuristic per tests/games/test_connect4_bench.py. So v1-c4
played at random-policy level. The format learned (zero
forfeits — every reply parseable, every column legal); the
strategy did not.
Suspected root causes, ranked:
- Corpus signal too weak. 47% of training rows were the “search” tier with generic rationales like “playing column N based on a depth-3 evaluation.” Generic text gives the model nothing to anchor learning on.
- Depth-3 minimax is too shallow for Connect4. Real strategy needs depth 6-8 to see threats developing two moves out.
- Mirror leakage in the val split. ~95% of val rows had their L-R mirror in train, so val loss converged to 0.094 on shape-memorization, not generalization. The val curve was uninformative as a stopping signal.
- No game history in prompts.
connect4_bridge.render_promptshows only the current board, not how threats developed.
Memory note saved at project_v1_c4_results.md so v2-c4’s
corpus design starts from “here’s what didn’t work” instead of
re-deriving it from cold logs in three weeks.
Deploy decision: don’t wire v1-c4. The React UI keeps the heuristic-fallback path. v1-c4 stays on disk for v2 comparison. Don’t ship a model that plays worse than what’s already running.
A bench-debug fix shipped along the way: bench_v1_c4.py v0
wrapped MLXBackend.generate (async) inside asyncio.run()
from a sync Policy callback. That deadlocked on the first
call’s threadpool executor cleanup — process sat at 5.5% CPU,
0.8% MEM for 43 minutes before user Ctrl-C’d. Fix
(a885e5a): skip the pitts async layer entirely, call
mlx_lm.load and mlx_lm.generate directly. Plus per-game
progress prints (the previous version was silent for 43 min;
the new version shows running win-rate after every game),
warmup phase before run_match starts, and HF_HUB_OFFLINE=1
baked in so the bench never hits the HuggingFace Hub API.
Afternoon — Sprint S.1: agent-driven security hardening
Pre-orchestration security audit. The framing in BACKLOG section 0: every Critical finding widens the moment background agents run unattended. Today they’re mitigated by “Jason is at the keyboard.” Orchestration breaks that mitigation by definition.
Black-Team pass #1
Surfaced 1 Critical + 4 High + 5 Medium + 4 Low (14
findings). The critical was load-bearing: werbos serve defaulted
to LEASH_YOLO, so every chat turn over
/api/chats/{id}/messages could call run_command with
arbitrary shell as the user, with NO confirm gate. Reachable via
prompt-injection in any model context — fetched URLs, OCR’d
images, MCP-bridged turns, memory files. Background agents would
make this catastrophic; today it’s “the user is watching.”
Black-Team pass #2 — verification + delta
Verified all 14 still applied. Found 4 new lower-tier issues:
the _PinnedHTTPSConnection SSRF guard ignores HTTPS_PROXY env
vars (defense-in-depth gap), TLS handshake failure leaks raw
TCP sockets, lazy backend init has a one-layer-deeper race past
the asyncio.Lock, and the React payload’s reasoning field is
unbounded. 18 total findings.
Red-Team — 17 fixed in one commit (ddfd9a9)
Red-Team consumed both Black-Team reports and remediated all of them except L4 (deferred — active-session pagination shape may change in Sprint A.1). Highlights:
- C1.
werbos servedefault leash flipped toNone(no shell tool registered at all). Opt back in via new--shell-yoloflag with explicit acknowledgment that prompt-injection paths now reach the shell. CLI keepsLEASH_YOLOsince a human is at the keyboard. - H1. New
BodySizeLimitMiddlewarecaps inbound bodies at 50 MB. Affects every POST/PUT in the app. - H2.
/phone-api/<path>is off by default; behind a new--enable-phone-bridgeflag. When enabled, path/method/ header allow-lists drop Cookie / Authorization / X-Forwarded-* before forwarding. - H3. Module-scoped
asyncio.Semaphore(1)per game-type serializes concurrent generates; 429 on saturation. The heuristic-fallback path stays uncapped (pure Python). - H4.
--shell-yolodeployments default workdir to~/werbos/scratch/instead of git toplevel. Closes thecd ..; rm -rfescape vector. - M4.
_redact_secretsintools/code/_shell.pyreplaces Bearer tokens,api_key=...,hf_<30+>, AKIA keys with[REDACTED]before the runner’s stdout/stderr lands in the PFM archive. Stops accidentally leaking credentials into SFT-export corpus.
35 new tests pin each hardened surface. 1029 → 1064 passing.
The pattern itself — Black-Team finds, Red-Team fixes, I review
- commit — is genuinely useful as a multi-agent workflow. Three Claude sub-agents, one human reviewer, one shipped sprint. The shape will generalize.
Late afternoon — flattening the god functions
Per the Archaeologist’s earlier finding, _build_handlers (TTT)
and _build_c4_handlers were 183 / 236 lines each — five
nested closures over a shared dict-of-Any state, the kind of
factory that’s hard to read in one pass.
Refactor (fd49543). Pulled out a typed _BackendState
dataclass. Lifted the four closures (_ensure_backend,
_generate_ttt_move, _generate_c4_move_lora,
_heuristic_c4_choice) to module scope where they take state
explicitly. The handler factories shrank to 72 / 105 lines —
each one is now just two clean route handlers back-to-back, not
nested-closures-over-shared-state. Cognitive load way down.
Bundled a small Archaeologist quick-win: dropped the hardcoded
backend="react+mlx" default in the generic
werbos/server/game_archive.py so it doesn’t leak its first
caller’s runtime into module-level defaults.
Ran the >80-line god-function detector again afterward.
_build_handlers fell off the list entirely; _build_c4_handlers
shrunk by 131 lines. Of the 31 functions still over 80 lines,
9 are argparse add_subparser blocks (long but cohesive),
5 are protocol-shape builders the Archaeologist explicitly
called out as durable, and 17 are real refactor candidates —
mostly in werbos/session/chat.py (step_stream,
_run_vision_interlude, three copies of _do_ai_turn).
Archaeologist’s call on the _do_ai_turn triplet: “Three
copies is the threshold for extraction, but only after the
third copy stabilizes.” Wait until Sprint A.1 adds a fourth
and fifth copy; the right shape will be obvious then.
Evening — werbos becomes an observe consumer
The orchestration-plane pivot. User confirmed they were ready to start Sprint A.1, NOT carry the v1-c4 disappointment forward.
Read both sibling SPECs end-to-end: ~/Documents/Companies/ werbosLLM/observe/SPEC.md (DRAFT-1, see-layer protocol —
trigger sources, predicate language, foreground-busy interface,
webhook HMAC) and ~/Documents/Companies/werbosLLM/watchtower/ SPEC.md (DRAFT-3, summon-layer — Triggers, Dispatches,
PendingActions, audit log).
The discovery: observe specifies the foreground-busy interface as an in-process Python callable, not a wire endpoint. Cross-process is the consumer’s concern. So werbos ships two layers — a pure-Python adapter for embedded consumers and an HTTP wrapper for cross-process polling.
A.1 layer 1 — pure-Python adapter (7d6a9ea)
SessionRegistry.is_foreground_busy() reads per-session
lock.locked() state. The lock is acquired by step(),
step_stream(), and close() and held for the duration of
the AI turn — so “lock held” is the canonical “AI turn in
flight” signal werbos already maintains.
werbos.observe.make_foreground_busy_callable(registry) wraps
that method in the exact Callable[[], bool] shape observe
SPEC v1 § 4 specifies: synchronous, side-effect-free, returns
in microseconds. The closure captures the registry by
reference — no caching, no construction-time snapshot — so
sessions added or closed after the closure was built are still
seen on subsequent calls. 10 new tests pin all of this.
A.1 layer 2 — HTTP endpoint (b90eb66)
GET /api/observe/foreground-busy
{
"protocol": "OBSERVE-PROTO-v1",
"foregroundBusy": false,
"asOf": "2026-05-08T00:56:09.349774Z"
}
The response shape is observe-conformant where the spec applies:
OBSERVE-PROTO-v1 magic, camelCase wire, ISO-8601 UTC with
trailing Z (not +00:00), the {error, detail} envelope on
failure paths. The route shape itself is werbos’s choice —
observe v1 doesn’t define a wire endpoint for foreground-busy,
so werbos owns /api/observe/* as the prefix where future
observe-shaped routes (state document, watchtower webhook
receiver) will land.
Werbos can now SAY it’s busy. It can’t yet HEAR triggers, ENQUEUE work, or RUN background agents — that’s Sprint A.2, queued for tonight. But the see-layer adapter is in.
Night shift — the games become the corpus
Night-shift framing in one line: the morning shipped two dogfood loops (TTT + Connect4). The night shift turned that pattern into a parallel dogfood factory — five more playable games, each with the per-move rating UI baked in — plus the orchestration plumbing (A.2), the per-repo context layer (K.1), the streaming polish (U.1), and the local-environment doctor (T.1) that the factory will need to actually run.
Mid-session aphorism the user gave when committing the games wire: “that will dogfood logic and turn into the deep reasoning and logic model and adapters.” The lock-in is intentional — each game is a different shape of bounded-rationality problem (short tactical / hidden-info / long-horizon / deception) and each one gets its own labeled-blunder corpus once dogfood mode is on.
A.2 — orchestration plane online (five slices)
The “user said they’d pick this up tonight” prediction from the evening section. They did.
- Slice 1: SQLite-backed task store (
710b5d6).werbos.tasks.TaskStorewith optimistic-lock CAS via aversioncolumn, status transition rules (pending → running → done|failed|cancelled), UUIDv7 ids for time-sortable enumeration, andtenant_id/user_idcolumns baked in from day 1 — the gatekeep multi-tenant seam per BACKLOG § 0.SSO. Single-user mode defaults"default"/"local"so today’s behavior is unchanged. - Slice 2: background work-queue runner (
e888647).werbos.tasks.TaskRunneris an asyncio loop in the Starlette lifespan that yields to the foreground via observe’sis_foreground_busy()callable from A.1 — when the user is mid-turn, the runner stays out of the way. Dispatches each task to a registered Executor; failure path captures exception type + message into the task row instead of crashing the loop. - Slice 3:
/api/tasksCRUD endpoints (704fa9b). GET / POST / GET-by-id with the observe-shaped error envelope ({error, detail}+ ISO-8601-Z timestamps) and server-side tenant scoping. The HTTP surface is tenant- agnostic to the consumer — the gatekeep JWT will fill thetenant_idclaim later without changing client code. - Slice 4: stats-refresh executor (
6dd1969). First real Executor: rebuildswerbos stats-style aggregates off-thread when a chat archive lands. Lightweight enough to prove the runner end-to-end without needing the dogfood- replay agent shipped first. - Slice 5: chat slash-command dispatch (
ff45303)./tasksand/cancel <id>route through the chat surface to the task store. Closes the loop: a chat turn can enqueue a background task; the runner picks it up next idle moment; a future chat turn can ask for status. 59 tests across the five slices.
K.1 — .werbos.md per-repo context (d1e5c6e)
Smallest, shippable-solo Phase 9 layer. When werbos chat or
werbos serve boots, walk parents from cwd (or --workdir)
up to $HOME for .werbos.md; prepend the nearest hit to the
system prompt under a new # Project section between
# Persona and # Memory. Mirrors Claude Code’s CLAUDE.md
pattern. 50 KB cap; oversized files skipped.
werbos init stamps out a starter file: auto-detects language
- likely build command from common manifests (
pyproject.toml→ Python +pytest -q,package.json→ npm test,Cargo.toml→ cargo test, Makefile →make test), three prompts,--yesfor non-interactive defaults,--forceto overwrite. Tests: 1174 → 1228 (+54).
The architectural shape: ContextBuilder.build(..., project_context: str = "") is stateless and caller-pushed,
matching the T7 contract for memory_text. ChatSession caches
the discovered text on construction; future /reload-context
slash command can reassign the attribute.
U.1 — streaming polish across three surfaces
Three slices closed Sprint U.1 except for Phase 5 (per-vision labels in Rate UI, queued).
- Mid-stream JSON-tool-call swallower (
28411cc). Models without native function-calling sometimes narrate tool calls as raw JSON inside their prose (Let me check. {"name": "web_search", ...}). Pitts already salvages these at end-of-stream so the call fires correctly, but the JSON was flashing in the chat bubble before the post-stream refresh swapped it out. Newwerbos/session/_stream_filter.pyruns a stateful NORMAL → BUFFERING → SUPPRESSING machine with JSON-string- aware brace tracking (so{"q": "find }} this"}doesn’t exit suppression early) and leftover-text re-feed (so prose after the closing}survives —Before. {...} After.preserves “After.”). Confirmation triggers a singlePhaseEvent("tool_pending", "calling <name>...")that the UI’s existing phase-label path picks up generically. 21 tests across unit + integration. - CLI streaming parity (
95a7c6a).werbos chatREPL now streams tokens to stdout in real time. NewChatSession.run_stream(on_token, on_phase)is the streaming twin ofrun(); newmake_streaming_status_wrapperscancels the duck/status loop on the FIRST stream event (token or phase) instead of waiting for the full assistant message.werbos>prefix prints once per turn, then tokens stream directly; phase events render as inline[<message>]lines. Empty-prose iterations fall back to bulk-print so the user always sees something. APK + browser + terminal all share one streaming UX now. 17 new tests. - Vision phase-1–4 markers + persona rule cleanup
(
0b7a801). Audited the roadmap against actual code: vision-as-internal-context phases 1 (VisionContextProvider), 2 (_run_vision_interludein step_stream), 3 (IMAGE_UPLOAD_ENABLED = truein ChatView), 4 (PhaseEvent("vision" / "thinking" / "vision_failed")), and the--vision-model/--vision-host/--vision-promptCLI flags all already shipped earlier. Markers flipped from[planned]to[shipped]. Phase 5 (per-vision Rate UI labels, ~120 lines) stays planned. Persona discipline rule added todefault.mdv1.2.0: “To call a tool, use the structured tool-call channel your runtime provides. Never write a raw{"name": "...", "arguments": {...}}JSON object as the body of a chat reply…” Defense in depth alongside the new mid-stream filter — the filter catches it post-hoc, the persona rule asks the model not to do it in the first place.
T.1 — werbos doctor (d2bec30)
First feature of Sprint T.1 (CLI ergonomics). Five concurrent checks of the local environment, ANSI-colorized green/red checklist, exit-1 on any FAIL so shell glue can branch on it:
- ollama — daemon reachable on
--host? - models — expected tags (default
llama3.1:8b) pulled? - sibling-shas — local HEAD of each sibling repo matches
the SHA pinned in
pyproject [siblings]? Walks../and matches byremote.origin.urlso name mismatches like P.F.M. ↔ pfm/ resolve cleanly. - venv-editable — which siblings (and werbos itself) are editable installs?
- tailscale — backend Running per
tailscale status --json; warn-soft if the binary is missing.
637 lines, stdlib-only (urllib + subprocess + tomllib). 27
tests. Replaces a cluster of manual checks the user was running
by hand every time werbos chat misbehaved.
Game suite — five new playable games, all rating-instrumented
Each game ships with its own MoveList rating UI mirroring the per-move thumbs + note pattern from B.1. Server-side archive support is incremental (still landing per game) — the rating sidecars work today, the PFM archives follow. The point is to get the surface right so each game can be its own dogfood stream.
- Blackjack + chess (
f9360e3). Blackjack: 0–5 AI players + dealer, S17, BJ pays 3:2, double on first decision only, AI-seat personalities (basic strategy / tight / loose / mirror) auto-step at ~550 ms. Per-session bankroll + W/L/P tally. Chess: 8×8 with full legal-move generation — castling, en passant, auto-queen promotion, check/checkmate/stalemate. Engine inlib/chess.tsas pure functions;ChessViewis rendering + interaction only. Chess engine is uniform-random for v1; per the companion-mode memory, real engine integration is future work (chess is companion mode, not training). - Slot reshuffle (
5f43633). The game-suite picker’s Risk slot got swapped for Battleship / Checkers / Poker stubs — Risk is locked late in the v5+ ladder per the game-training roadmap, doesn’t make sense as the next slot. - Battleship + checkers (
300fbbd). Battleship: 10×10 dual grid, random fleet placement with reshuffle pre-first- shot, probabilistic-targeting AI that scores every unfired cell by the count of valid remaining-ship placements that would cover it; in target mode (open hits exist) restricts scoring to placements touching an open hit. Sunk-ship attribution by walking the contiguous hit line. Checkers: standard American/English rules, 8×8 dark-square play, forced captures with multi-jump chains, mid-chain promotion stops the chain, no flying kings. Multi-jump chains execute in one click on the final landing (best match by capture count when multiple chains end at the same square). Both engines pure functions inlib/for testability. - Heads-up Texas hold’em (
efb9768). You vs one AI. Standard pre-flop → flop → turn → river → showdown. Fixed 10/20 blinds, 1000-each starting stacks, button alternates per hand (heads-up rules: button is small blind; pre-flop button acts first, post-flop non-button first). Pure 5-card hand evaluator (high-card through straight-flush, wheel A-2-3-4-5 included) plus best-of-7 enumeration over the 21 5-combos. AI is hand-strength heuristic with ~8% bluff and ~5% trap noise so it isn’t transparent. Action bar: Fold / Check-or-Call / Min raise / Pot raise / All-in. Per-decision rating UI mirrors the other games.
UI polish + character file
Two smaller landings worth marking:
- Image thumbs + code-block rendering in chat bubbles
(
bb48dba). Bubbles now show inline thumbnails for uploaded images, and triple-backtick code fences render with a monospace block + correct whitespace. The vision loop’s “i think that is a glass of beer” moment from Day 5 is now visually grounded — the image actually appears in-bubble. - Game suite picker (
3f104f8). The five new games needed somewhere to live. Picker shows shipped games (TTT, Connect4) plus three coming-soon slots — the slots filled in across the night-shift commits as each game shipped. werbos-character.md— the 13% (f5f162a). Phase 10 groundwork: the character file (independent persona, what werbos won’t pretend to like, how it handles being wrong) separate from the voice-matching 87%. Captures the load-bearing distinction — character is constant across users; voice-matching is per-user. “persona character boundaries” in the slider design get their concrete file reference.
The night shift’s load-bearing decision: don’t gate any of this on agent execution. A.2 ships with one stub executor (stats-refresh); the dogfood-replay agent that the platform is for is queued, not blocked. K.1 ships project-context discovery with no slash-command reload — that’s deferred too. U.1 ships the JSON filter + CLI streaming + persona rule without Phase 5 vision labels. Each layer is a complete slice of its own; nothing is half-shipped behind a feature flag.
Numbers worth keeping
- 35 commits, 1266 passing tests (+357 since yesterday; +186 in the night shift alone over 1080).
- 18 security findings → 17 fixed in one Red-Team commit; L4 deferred per audit instructions.
_build_c4_handlers: 236 → 105 lines. TTT: 183 → 72.- v1-c4 vs heuristic: 3% as O. Random baseline 0-3%.
- Heuristic vs random in our smoke run: 100%. v1-c4 performed effectively at random level.
- Black-Team agent runs: pass #1 found 14, pass #2 confirmed all 14 + found 4 new = 18.
/api/observe/foreground-busyround-trip: <5 ms with no sessions. The microsecond claim from observe SPEC § 4 holds.- A.2 task store: 59 tests across five slices. SQLite- backed, tenant-scoped, observe-shaped error envelope.
- K.1: 54 new tests for
discover_project_context,ContextBuilderthreading,ChatSessionintegration, andwerbos initauto-detection. - U.1 mid-stream filter: 21 tests pinning state-machine
transitions, JSON-string-aware brace tracking, leftover
re-feed, runaway / DoneEvent fallbacks. CLI streaming
parity: 17 tests across
run_streamcallbacks + the streaming wrappers. - Game suite: 5 new playable games. Blackjack (with AI seat personalities), chess (random-mover), battleship (probabilistic-targeting AI), checkers (forced captures, multi-jump chains), heads-up Texas hold’em (5-card eval with bluff/trap noise).
werbos doctor: 5 checks, 27 tests. Stdlib-only.
AHAs
The infrastructure is the artifact
v1-c4 failed. The corpus generator did not. The bench harness
did not. The Connect4 React UI with heuristic fallback did not.
The werbos/server/game_archive.py extraction did not.
Tomorrow’s v2-c4 corpus is a git diff on
scripts/build_c4_corpus.py away. The point of an
infrastructure-first day is that the next iteration is small.
Mirror leakage inflates val loss
95% of val rows had their L-R mirror twin in train, so val loss
collapsed to 0.094 on shape memorization, not generalized
skill. The fix is stratified splits that keep mirror pairs
co-located — a one-flag addition to split_corpus.py. v2-c4’s
val curve will be a real generalization signal.
Depth of oracle / game complexity is the load-bearing variable
TTT had cheap full-tree minimax; v5-ttt got +8.5pp. Connect4’s depth-3 minimax was barely past tactical; v1-c4 got effectively zero. Either go to depth 6-8 for v2-c4 (cost ~117K leaves per position, but corpus generation is one-shot) or mine real labeled blunders from the dogfood-replay agent once it ships. Probably both.
Agent-driven security audits work
Black-Team pass #1 surfaced 14 findings. Pass #2 verified them
- caught 4 more. Red-Team consumed both reports + fixed 17 in one commit. Three Claude sub-agents, one human review, one shipped sprint. The pattern generalizes — same shape will work for compliance audits, performance regressions, dependency updates, etc. The orchestration story I’m now writing in werbos.observe + watchtower is the same story I’m already running, just with sub-agents instead of background tasks.
Three copies isn’t enough to abstract
Archaeologist’s call on _do_ai_turn (chat / partyparrot /
tictactoe — three copies of the same 60-line spine): “Three
copies is the threshold for extraction, but only after the
third copy stabilizes.” Wait until Sprint A.1’s
background-agent _do_ai_turn exists, then extract with 4-6
call sites voting on the abstraction’s shape. Refactoring
against three copies that are about to be six is doing the
work twice.
The games surface IS the corpus surface
The morning shipped TTT + Connect4 each as one dogfood loop. The night shift made the factory — five more games (blackjack, chess, battleship, checkers, hold’em), each with its own MoveList rating UI baked in from day 1. That isn’t five UI features; that’s five parallel labeled-blunder pipelines waiting for the dogfood-replay agent to start consuming them. Each game is a different shape of bounded-rationality problem (short tactical / hidden-info / long-horizon / deception), and each one feeds a different reasoning specialist later. v5+ ladder stays locked (TTT → Connect4 → blackjack → battleship → checkers → poker → Risk; chess is companion-mode), but the order is the training order, not the build order. Build them all up front so the corpus dataframe exists across the matrix; train against whichever cell has the cleanest labeled signal first.
The mid-stream JSON filter is its own bench
21 unit tests pin a tiny state machine — NORMAL → BUFFERING →
SUPPRESSING — with three load-bearing properties: JSON-string-
aware brace tracking (so {"q": "find }} this"} doesn’t exit
suppression early), leftover-text re-feed (so Before. {...} After. preserves “After.”), and DoneEvent flushing (so a
malformed JSON at end-of-stream surfaces as text instead of
silently disappearing). The persona rule (v1.2.0 of
default.md) tells the model not to emit JSON-as-prose;
the filter is the actual enforcement. Stock 8B models don’t
always honor “never,” so defense-in-depth is not optional —
the filter ships, and the persona note rides along.
CLI streaming parity = three surfaces, one experience
werbos chat, werbos serve (browser), and the APK now all
stream tokens in real time. The pattern: status loop cancels on
the FIRST stream event (token or phase) instead of waiting for
the full assistant message. werbos> prefix prints once per
turn, then tokens stream directly. Phase events render as
inline [<message>] lines so tool-pending / vision /
vision_failed all surface the same way across surfaces.
Implementation gravitates to one shape: ChatSession.run_stream( on_token, on_phase) is the streaming twin of run(); the new
make_streaming_status_wrappers returns
(human_turn, on_token, on_phase, on_assistant) instead of
the old (human_turn, on_assistant). Three callbacks, one
event-driven state machine, one prefix-printed-once invariant.
.werbos.md is the cheapest leverage in Phase 9
K.1 was 1–2 hours per the roadmap; landed in two. The repo-
context layer that every chat session now sees, sibling-repo
relationships and “don’t touch X” warnings and build/test
commands, all without any per-session manual injection. Walk-up
discovery means it Just Works in any subdirectory; werbos init
auto-detects from manifests so a new repo is one command away
from instrumented. The K.2 user-memory layer is bigger, the K.3
proactive-surfacing layer is the endgame, but K.1 is the layer
where the marginal cost was a coffee and the marginal benefit
shows up in every session. Ship the cheap stuff first.
Don’t gate slices on agents that aren’t built yet
A.2’s executor today is stats-refresh — a stub. The dogfood-
replay agent that A.2 exists for is queued, not blocked. K.1
ships discovery without the slash-command reload. U.1 ships
JSON filter + CLI streaming + persona rule without Phase 5
vision labels. Each layer is a complete slice on its own;
nothing is half-shipped behind a feature flag. Velocity comes
from accepting that the next layer will need rework, not from
trying to design the right shape for everything in one pass.
What’s queued
- Phase 5 vision labels in Rate UI — per-vision good/bad thumbs on the vision-observation events. ~120 lines, ~2 hrs. Last U.1 item.
- Watchtower Dispatch consumer — observe SPEC § 2.4.1
webhook with HMAC verification. Wired into the
/api/observe/*prefix that A.1 layer 2 already established. A.2 + foreground-busy now both shipped, so this slice has the full upstream context it needs. - Game adapters / first new-game LoRAs. Each of the five new games is a fresh corpus path — blackjack basic-strategy blunders, chess companion-mode mining, battleship targeting decisions, checkers multi-jump chains, hold’em bluff/trap detection. v5+ ladder roadmap stays locked: TTT → Connect4 → blackjack → battleship → checkers → poker → Risk; chess remains companion-mode, not training.
- v2-c4 corpus at some later sweep. Drop the search-tier rows or label them with concrete tactical structure; deeper minimax (6-8); stratified split keeping mirrors co-located; possibly add game history to the prompt template.
_do_ai_turnextraction post-A.2 when 4–6 call sites exist. The third copy stabilized today; A.1’s background-agent path will be the fourth.- L4 — pagination on active-session
/api/runs/chat/{id}. S.1 deferred this pending A.1 shape. A.1 layer 2 shipped, so the active-session shape is now stable enough to revisit. /reload-contextslash command — small K.1 follow-up. Reassignssession.project_contextso a.werbos.mdedit takes effect without restarting the chat.- § 0.SSO → multi-tenant werbos — the gatekeep-paired
self-host SKU. Captured in BACKLOG, not on the work surface
yet.
tenant_id/user_idseam baked into A.2 so adding tenancy later is a default-value change, not a refactor.
Day 6 final tally: thirty-five commits, six sprints closed, five new playable games, +357 tests. Started at “i like winning but i like better training data” (post-v1-c4 bench), ended at “the games become the corpus.” Hell of a day.