Start here

Overview

The coding-agent quality-control loop, and the line it never crosses.

EvalGlass is an AI quality-control tool that lets your coding agent build and run project-specific checks for your AI application. It reads real behavior, shapes checks from it, runs locally, and returns a scorecard. It measures and reports; you decide and gate. Runs locally, keeps project truth in your repo, no false green.

EvalGlass does not certify, guarantee, or approve anything. The guarantee it makes is narrower and more honest: given the evidence, authority records, metrics, thresholds, calibration, baselines, and data policy you supplied, it emits exactly the Scorecard and verdict it can stand behind — and nothing beyond it. A green result is bounded by that sentence.

HOST-OWNED EVIDENCE

you own the truth

datasets

exported traces

measure

EVALGLASS

one verdict path

measure

resolve authority

Verdict Engine decides

emit

SCORECARD + RUNRECORD

the bounded claim

informational by default

gates only if authorised

host-owned
EvalGlass
bounded claim

EvalGlass turns host-owned evidence into a bounded claim. Only the Verdict Engine decides the outcome, and a Scorecard stays informational by default; green is reserved for a specific approved pass.

The loop

Evaluation here is a loop you run with your coding agent, not a one-off audit:

Step	What happens
Ask	You make a natural-language request to your coding agent — “evaluate my app after the model switch.”
Observe	EvalGlass reads host-owned evidence: repo, exported traces, prompts, LLM calls, examples, releases.
Shape	It scaffolds proposed project-specific checks and scenarios — it shapes them, it does not generate authoritative gold.
Run	The vendored runtime runs locally — no platform, no keys, no network.
Read	You read a scorecard: values, deltas, unknowns, diagnostics, and the bounded verdict.
Improve	You compare, fix, review, or promote a gate and wire CI.

What runs where

Layer	Role	Does not do
Coding agent	Reads, scaffolds, runs commands, explains.	Does not grant authority.
EvalGlass runtime	Computes typed scorecards and verdicts.	Does not edit your app’s behavior.
Host repo	Owns examples, traces, metrics, thresholds, baselines.	Does not get silently overwritten.
Extensions	Bring behavior in, or send scorecards out.	Do not create false authority.

The four run outcomes

Verdict	Means	CI
informational	Measured, but no gate is active.	exit 0 — does not fail the build
pass	Active gates met their approved thresholds.	exit 0
fail	An active gate was validly below threshold.	exit 1 — fails the build
blocked	An active gate could not be claimed honestly.	exit 1

A fresh run gates nothing, so it is informational and exits 0 — not pass. Green is reserved for a specific, approved pass; it never means “safe AI.” New to the model? Read the Claim Boundary — what a green result is and isn’t allowed to mean.

Install

EvalGlass ships as a Claude Code plugin — two commands, byte-identical wherever you run them:

bash

/plugin marketplace add Syntelesis-Lab/evalglass
/plugin install evalglass@syntelesis-lab

See Quickstart for a zero-install demo, or Install to adopt it in your repo. Codex is supported as a second runtime; its public marketplace listing is next, not yet published.

Pre-alpha, and capability is not authority EvalGlass is v0.1.0, pre-alpha, untagged — no PyPI release, Apache-2.0, local-first, no telemetry. That a check can run says nothing about whether it is allowed to gate: authority comes only from host-owned records you approve, never from the tool. That gap is the whole reason EvalGlass exists.

Next steps

Start here

Quickstart

One command to a first Scorecard.

Run and read

Read a Scorecard

Every field in the result, and what it can’t prove.

Reference

Agent commands

The /evalglass verbs your agent drives.