The coding agent as evaluation operator

If you build AI features, you almost certainly already drive a coding agent — Claude Code, or Codex — that reads your repository, edits files, and runs commands on your behalf. It is the obvious operator for evaluation work, because it already carries the context: it knows where your prompts live, which model you just swapped, and what a regression in your support flow would even look like. EvalGlass leans into that. You operate it through the agent, in plain language, in the same conversation where you made the change. But the moment a tool that produces evaluation evidence is also the actor that runs your build, a quiet question becomes load-bearing: who is allowed to decide that the evidence counts? This guide is about the line EvalGlass draws to answer it — the line between operating the work and owning the authority over what the work is allowed to claim.

The shorthand is controlled delegation. You delegate the operating work to the agent without delegating the authority. That distinction is not a slogan bolted on after the fact; it is wired into the verbs the plugin exposes, the typed states everything it authors lands in, and the parts of the system the agent simply has no command to touch.

Operator, not author

Start with what a coding agent is, by default, before EvalGlass is involved at all. It is an operator that already runs under a human-set permission model. Claude Code, in its default mode, pauses and asks you to approve before it edits a file or runs a command;¹ Codex similarly asks for approval to edit files outside the workspace or to run commands that need network access.² You can loosen those modes deliberately, granting the agent more autonomy for a session — but that is a choice you make about convenience, not a transfer of judgment. This matters because it is the most seductive failure in agent-operated evaluation: reading autonomy as authority. An agent that ran for twenty minutes without stopping to ask did not thereby earn the right to declare its own output trustworthy. "It didn't stop to ask" is a permission setting you chose; it is not a sign the result has been reviewed.

Capability is not authority. The actor that creates evidence cannot be the actor that authorizes it — for the same reason you would never let a function grade its own output.

EvalGlass takes the permission model these agents already ship with and extends it into the evaluation domain. Anthropic's own framing of safe agents is that humans should retain control over how goals are pursued, particularly before high-stakes decisions.³ Promoting a check to a build gate is exactly such a decision. So the design holds the agent in the operator's seat and keeps every act that confers authority on the human side of the line. The agent is, in the plugin's own words, a typist and a reader — it issues exact commands and reads back typed artifacts; it is never the participant that decides the verdict.

What the coding agent can do

Within that seat, the agent is genuinely useful, and it is worth being concrete about its reach rather than vague about its limits. It can inspect: run a read-only status overview, discover candidate call sites in your code (announced and consent-gated, not silently), and read repo context together with traces you have already exported — strictly from OpenTelemetry, OpenInference, or local JSON, with no SDK and no network on the run path. It can scaffold: draft a project-specific dataset, append a metric, or stand up an LLM-as-judge with a starter rubric. It can run and read: invoke the vendored runtime locally, surface per-metric values and status, and explain — from typed authority and diagnostics, not from its own opinion — why a given number is or is not trustworthy. And it can compare and suggest: line a run up against a baseline when, and only when, the runs are marked comparable, recommend the next check worth adding, and, at your explicit direction, type the host edits that would activate a gate you have decided to approve.

That last clause is where people misread the boundary, so it is worth slowing down. The agent can type the edit that turns a check into a gate. What it cannot do is supply the authority behind it. The keystrokes are operation; the decision is yours, recorded in your own config, in a human-reviewed file in your repo. The agent is a fast pair of hands on the keyboard, not a signatory.

What stays host-owned — and how

Five inputs confer authority, and all five stay with you and your repository. Each is not merely "owned by the human" as a matter of etiquette; each has an exact mechanism that the agent has no verb to perform. EvalGlass owns the evaluation methodology and the managed runtime — you own everything that decides what passes.

Notice the symmetry: every operating move the agent makes has an authority act sitting opposite it, and the agent's reach stops precisely at the line between them. It can scaffold a dataset; you validate it. It can suggest a threshold; you approve it. It can stand up a judge; you calibrate it. The full reference for this division lives at the vocabulary reference and the host-ownership reference in the docs.

The closed verb set

The cleanest way to see controlled delegation is to look at the commands themselves. Everything is operated through one umbrella, /evalglass, and the verbs are a short, closed list: setup and connect to install the runtime and point it at your traces; run to evaluate; view, explain, and compare to read results; baseline to manage what a run is measured against; and ci to wire evaluation into your pipeline — plus a tier of authoring helpers such as add-metric and add-judge. You do not have to memorize any of this. You describe what you want — "evaluate my assistant after the model switch" — and the right skill triggers; the verbs are what the agent reaches for, not what you type. The explicit names exist for when you would rather drive directly. The complete list, with each verb's contract, is in the agent commands reference.

The identity of this list is as much in what is absent as in what is present. There is deliberately no gate, no approve, and no certify verb. That is not an oversight waiting to be filled in a later release; it is the design. There is no command for the agent to call that would cross from the operator's column into the authority column, because the only way to confer authority is a host edit you make yourself. The absence of the verb is what keeps the operator from ever authoring authority.

Why scaffolding is not approval

The other half of controlled delegation is what happens to everything the agent creates. The rule is mechanical, not aspirational: each scaffolding verb writes exactly one draft typed state and nothing more. A dataset the agent authors lands proposed — explicitly not validated reference data. A metric it appends is informational — it reports, it does not gate. A judge it stands up is uncalibrated — advisory until you calibrate it against your own labels. And on a fresh install, authority.json starts empty, so the very first run is informational by construction: it reports what it measured and exits cleanly, deciding nothing, because no human has yet reviewed anything for it to stand on.

This is enforced down at the library, which is why it survives an agent that is trying to be helpful. Ask a benchmark to approve a threshold and you do not get an approval — the runtime raises a GovernanceError and refuses, because a benchmark is not an authority. Import synthetic data labeled "validated" and you still get a proposed dataset, because the label is not yours to grant by importing. The point is that generated does not mean approved, and the system declines to pretend otherwise even when asked nicely. The verb surfaces evidence; only a host edit you own re-types that evidence as authoritative. You can read the precise rule at host ownership.

Two runtimes, one model

Controlled delegation is a property of the design, not of any one host, so it holds across runtimes. Claude Code is the primary runtime — the one the operating model was built around. Codex is a supported second runtime, driving the same skills tree with the same verbs and the same authority boundary; the experience is the operating model, not the host. What is still to come is purely the storefront: a public Codex marketplace listing is a maintainer step that is not yet published, so today you install the Codex runtime directly rather than from a public listing. None of that changes the boundary — both runtimes are operators, and on both, authority stays with you. The setup specifics are on the Codex runtime page, and the broader picture of agent-operated evaluation is on Claude Code evals.

Step back and the shape is simple. You bring a coding agent you already trust to do work. You keep, for yourself, the small set of decisions that say what the work is allowed to mean. The plugin makes the first easy and the second unavoidable — and the gap between them is exactly where false confidence would otherwise creep in.

Sources

Coding agents already operate under a human-set permission model — the basis for treating the agent as operator and the human as authority. EvalGlass's specific design (a closed verb set with no gate, approve, or certify verb) is its own, not a claim made by the sources below.

1. Anthropic — Claude Code permission modes (accessed 2026): by default Claude Code pauses and asks you to approve the action before editing a file or running a command; other modes loosen this at your choice.
2. OpenAI — Codex agent approvals & security (accessed 2026): Codex asks for approval to edit files outside the workspace or to run commands that require network access, with read-only and broader modes you select.
3. Anthropic — Our framework for developing safe and trustworthy agents (2025): humans should retain control over how goals are pursued, particularly before high-stakes decisions.