werbos

How to rate

The Rate UI: a browser-based GUI for thumbs-up/thumbs-down on past chats. Faster than the CLI when you have a backlog.

The Rate UI is the visual companion to werbos label. Same labels, same sidecar JSON files — just a faster batch interface for when you have a stack of unlabeled archives to clear.

Start the UI

# terminal 1: backend
werbos serve

# terminal 2: React frontend
cd werbos/ui
npm install               # one-time
npm run dev

Open http://localhost:5173.

In production (Docker / built bundle), the UI is served from the same origin as the backend. In dev, Vite proxies /api → werbos serve on :8000.

What you see

Two-pane layout:

  • Left: list of chat + code archives, newest first. Each row shows the title (auto-derived from the first user message), date, and a verdict badge (green dot for good, red for bad, gray dash for skipped, empty for unlabeled).
  • Right: the full transcript of the selected archive, message-by- message, code blocks rendered.

Filter pills

Above the list, four filter pills:

  • All — everything
  • Unlabeled — only items needing your verdict (the action queue)
  • Good — your approvals
  • Bad — failures, useful to scan for patterns

Default is Unlabeled. Click an item, read it, click a verdict button, move on.

Per-archive actions

In the right pane:

  • Good — sets verdict to success
  • Bad — sets verdict to failure (asks for an optional note)
  • Skip — sets verdict to skipped
  • Clear verdict — removes the sidecar JSON

Writes happen immediately (no save button). Same sidecar files the CLI uses, identical schema.

Workflow that actually works

Once a week:

  1. Open the Rate UI on Unlabeled
  2. Speed-run the queue. Each archive takes <30s if you remember the session, longer if you don’t
  3. Unlabeled count drops to zero
  4. Run training on --label good

The UI exists because doing this from the CLI on 50+ archives is miserable. Doing it visually with the transcript right there is ~5x faster.

Multi-user (backlog)

Single-user today. Multi-user permissions on the rate selection is on the backlog — if multiple humans should label independently, each will need their own sidecar namespace. Not blocking solo use.

Same data, three tools

        +-- werbos label CLI --+
        |                       |
        +-> sidecar .label.json +
        |                       |
        +-- Rate UI ------------+
        |                       |
        +-- post-run prompt ----+

All three write the same file. Pick whichever fits the moment.

Next