werbos

How to nudge

The dogfood loop: label good and bad chats, filter to good, retrain. The model gets better as you use it.

“Nudging” is werbos’s name for the dogfood loop. Every chat session is archived as a PFM file. You label each one good / bad / skip. The exporter filters by label. The trainer learns from the good ones. Loop closes. Werbos gets better.

chat session  ->  .pfm archive  ->  label  ->  filtered SFT  ->  next adapter
   ^                                                                  |
   |                                                                  |
   +--- werbos uses the new adapter -----------------------------------+

After every session

werbos asks at the end of an interactive run:

CHAT OVER: ended (3 turn(s))
Archived to: ~/werbos/chats/abc123def.pfm
Verdict? (good/bad/skip):

Type one of:

  • good — the exchange went the way you wanted. Train on this.
  • bad — werbos messed up. Optionally add a one-line note.
  • skip — ambiguous, neutral, or you don’t care. Won’t be used in --label good filtered training.

Hit enter without typing to default to skip.

Labeling later (CLI)

Forgot to label? Or want to revise? The werbos label command:

# label one
werbos label abc123def --good
werbos label abc123def --bad --note "looped on tool calls 4x"
werbos label abc123def --skip
werbos label abc123def --clear   # remove the verdict

# label by glob
werbos label "$(ls -t ~/werbos/chats/ | head -1)" --good   # last one

The verdict lives in a sidecar JSON next to the archive: ~/werbos/chats/abc123def.label.json. Plain JSON; you can edit it manually if you really want to.

Labeling via GUI

If you’d rather click than type, see How to rate for the React UI — thumbs up / thumbs down / notes / browse history.

Filtering by label at export time

werbos export sft \
  --src ~/werbos/chats/ \
  --out ~/werbos/sft-good-only.jsonl \
  --label good

The --label filter is the lever:

FilterUse when
goodnormal positive-example training
badtraining a classifier on what NOT to do (advanced)
labeledyou’ve reviewed everything; both polarities matter
unlabeledcatch up on review queue
anydump everything (default; not recommended for production training)

What to label good vs bad

Concrete signals:

good if:

  • werbos answered correctly and concisely
  • werbos asked the right clarifying question
  • werbos used tools without looping or fabricating
  • the exchange is something you’d want werbos to imitate next time

bad if:

  • werbos hallucinated facts you can verify are wrong
  • werbos looped on tool calls (the v1 failure mode)
  • werbos refused something benign with safety boilerplate
  • werbos forgot context from earlier in the same session
  • werbos was sycophantic (“Great question!”)

skip if:

  • the conversation was you testing or exploring, not real work
  • the topic is so unique you wouldn’t want werbos to generalize it

The compound effect

A single label means nothing. Hundreds of labels become a strong training signal. After a month of normal use, you typically have:

  • ~50-200 good archives (per use frequency)
  • ~10-30 bad archives (failures stand out, easy to label)
  • the rest unlabeled (skipped or untouched)

Filter to good → train v(N+1). Bench it. Promote if it wins. Repeat.

What to never label

Don’t label in werbos’s own dogfood project (transcripts of you building werbos). The importer skips werbosLLM by default precisely to avoid circular hallucination — werbos training on werbos’s critique of werbos.

Next