You ask. Your agent runs the loop. You decide what gates.

The shape of a run

Your system in, a scorecard you own out.

The loop

One loop your agent runs — and each step is a command.

1. 1

   Ask

   Ask a normal product question in plain language: evaluate this app, compare this model, explain this drift.

   no command · you just ask
2. 2

   Observe

   It reads repo context, prompts, model calls, tools, and your exported traces — read-only, no network.

   command setup connect
3. 3

   Shape

   It authors project-specific checks — metrics, rubrics, scenarios, judges, baselines — each landing proposed, never approved.

   command add-metric add-judge calibrate
4. 4

   Run

   The vendored runtime evaluates behavior from that evidence, locally. The plugin never computes a verdict itself.

   command run
5. 5

   Read

   The scorecard explains every value, delta, and authority state — and says plainly what the run does not claim.

   command view explain
6. 6

   Cluster next

   Group failing and non-scored items by shared cause, so a flat number becomes one actionable failure mode.

   shipping next
7. 7

   Watch next

   Re-run on a schedule and compare to your baseline — a typed drift.json from cron or CI, never a daemon.

   command watch
8. 8

   Improve

   Fix prompts, switch models, add examples, tune metrics — or wire CI to gate on a threshold you approve.

   command compare ci
9. ↻

   — and back to Ask. The loop repeats as your app changes; each pass leaves the suite a little stronger.

Everything the agent proposes lands proposed and informational — it suggests what to measure, never approves it. A fresh run reports; it doesn’t fail your build until you promote a gate on a threshold you approve. Every verb & prompt, in the docs →

Who holds what

Four roles — and only one of them decides.

The wider loop

Core runs it. Add the next product only when the problem changes.

Everything above is Core — open source, in your repo, today. Two commercial products extend the same loop when the question changes: Discovery when you need the checks your suite is not even measuring yet, and Intelligence when a failure is real but you cannot yet say why. Each hands back ordinary Core artifacts you keep.

Discovery and Intelligence are separate, paid, pre-alpha products — neither is generally available, and neither is shown running here. Compare all three →