Day 38 — 'the forest, and the first memory.'
It started with a 4 MB proof — a model trained from random weights that only knows one project — and turned into a whole architecture. Wrote down the micro-model forest: a forest of tiny content-addressed models (the Merkle Mini Model), an SVO grammar that routes a prompt to the brain that knows, System-1/System-2 reflexes for ms latency, and PS2-style frustum streaming so the forest runs on hardware that can't hold it. Then werbos got an inner life: IM.1 gives it a first-person inner monologue on a timer (a 'Mind' page you can scroll like a therapist's notes), and the chat→memory harvester finally closes the 'remember' gap — werbos reads a conversation and writes down the durable facts about you. The harvester worked (it learned the right things from a real chat) and then taught the day's hardest lesson on 10 live conversations: unsupervised, a 3B writes ~30% junk and even a contradiction. Self-improvement without a gate tears DOWN. Also: /calc, because local models can't do arithmetic and shouldn't pretend to; and traces now show the request text by joining the experience log. The AHA: a personal AI that talks to you all day but never remembers isn't yours yet — and the thing that makes remembering safe is the gate, not the model.
“the forest, and the first memory.”
Two threads today, and they’re the same thread. One is a vision — the architecture for why small models win. The other is werbos starting to have an inner life — it learned to think on a timer, and to remember what you tell it. Both come back to the same bet: lots of small things you own beat one big thing you rent.
A real /calc (951a5b2)
Small one first. Asked werbos “what is 100 − 50?” and the tracer labelled
it a how-to. There was no math skill — only tip-calc/age-calc. And
local 8B models hallucinate arithmetic anyway, so a prompt-based math
skill would just be confidently wrong. So /calc bypasses the model
entirely (same pattern as /password-gen): it extracts the expression
from your phrasing — words and all, “100 minus 50” → 50 — and evaluates
it with a whitelisted AST walker. Never eval(); names, calls, and
exponent-bombs are rejected. The model is the cursor; arithmetic is the
calculator. 34 tests, graded in 3 ms.
The forest, written down (5f5a4a8 · df39d60 · 8be4f61)
This is the headline, and it started as a proof: a model trained from random weights on one project’s synthetic corpus — ~4.2M params, 16 MB, coherent within its lane, and architecturally unable to talk about anything else (it can’t spell what it never saw). It answered faster than werbos. That tiny win is the whole thesis in miniature, so I wrote the thesis down.
docs/vision/micro-model-forest.md — werbos isn’t one model, it’s a
factory for tiny single-purpose ones, plus a content-addressed
routing grammar over a forest of them. Skills are the verbs (what
it can do — already shipped), brains are the nouns (what it knows —
the new half), and an SVO parse is the grammar that composes them.
Each leaf is addressed by a hash of (corpus + weights) — Mixture-of-
Experts you can audit, delete, and sell.
docs/vision/hot-swap-ps2-inspired-memory.md — the latency half.
The MMM (Merkle Mini Model) gives werbos reflexes: System 1 (a
hash-walk to a resident mini-leaf = milliseconds) for the common case,
System 2 (the slow generalist) only for the rare hard question. And the
residency problem — you can’t hold the whole forest in RAM — gets the
oldest trick in real-time graphics: don’t load the world, load the
frustum. Frustum-cull the leaves you’ve moved past, stream in the one
the conversation is walking toward (predicted by the same loop the
autopilot already runs). A Pi streams a forest like a PS2 streamed
Shadow of the Colossus. The compute thesis underneath: frontier
re-derives the whole world every training run; the forest spends compute
only where a gradeable leaf failed — and for code, the grader is free.
The day’s docs/architecture.md pass (8be4f61) caught the rest up:
the autopilot subsystem, the debug surfaces, and a skill catalog that
crossed ~294.
IM.1 — werbos thinks out loud (3e825d3 · queued 43c6f45)
The autopilot already learns from its experience log. IM.1 lets it
narrate it. On a timer (--inner-monologue 5|10|30|60, opt-in) werbos
reflects over what it’s done and writes ONE short first-person thought
to a dedicated stream — and a dashboard Mind page renders the
timeline, each thought with a “why this thought” drawer linking the
traces it reflected on. It reuses the existing reflect() engine and the
autopilot’s daemon shape, so it’s mostly wiring. First real thought,
unprompted: “I’m still stuck in this narrow scope… maybe I need to
develop specific skills for those topics.” It wrote itself a to-do.
11 tests; live on the homelab, thinking every 30 minutes.
The request, back in the trace (b3a7dc8)
Asked werbos something via the autopilot, opened the trace, and the
prompt wasn’t there — just user_len: 13. By design: traces store only
lengths, never text (privacy + a tiny log). But the actual request
lives in the experience log, joined by trace_id. So /api/traces/{id}
now joins it — an autopilot trace shows what was asked (request +
skill + verdict) in a banner above the timeline, instead of sending you
to dig. Closes the “why don’t I see my prompt” gap.
The first memory — and the gate (5f2a18c)
The big one. werbos reads memory into a chat, but nothing ever wrote
memory out of one — so it would talk with you all day and forget every
word. The chat→memory harvester closes that: read a .pfm
transcript, ask the model for the durable facts about the operator
(goals, projects, role, relationships — refusing transient chitchat),
dedupe against what’s known, write them as memory cards with provenance.
werbos memory harvest, dry-run by default.
And it worked — on a real conversation it pulled exactly the right
facts (a SCORM-authoring project, that I built werbos itself, that I want
to be an AI architect). Then I ran it on 10 live chats with --write,
and it taught the day’s hardest lesson: unsupervised, a 3B is ~30%
precision. It wrote junk (joke-tell, build-debug near-dups the dedupe
missed) and even a contradiction — “currently an AI architect,” lifted
from a mock-interview chat, against the real goal of becoming one. Left
on, it would have quietly made werbos worse.
Which is the whole point, and the answer to the question I keep circling — does a self-modifying system build up or tear down? Without a gate: down. The fix is the pattern werbos already lives by: propose, don’t auto-write. Harvest proposes; you approve on the Memory page; only then does it remember. That’s the champion/challenger gate, applied to memory — and it’s what makes the loop safe to run on every conversation.
The AHA
A personal AI that talks to you all day but never remembers isn’t yours yet. Today werbos started to remember — and the thing that makes remembering safe turned out to be the gate, not the model. Build the bridge, then build the rail. The loop teaches you the rail by tearing something down once.
5698 green by end of day.