Runtime & Integrations · Runtime Harness

Reports & Exits

Render the Scorecard — never recompute it.

MarkdownScoreSink and TerminalScoreSink render a Scorecard to text; the verdict word comes straight from scorecard.verdict.verdict (no hardcoded token). Sinks do no I/O — the CLI writes the string — and the exit class is a fixed lookup over the verdict.

A report is the most tempting place for authority to leak in as prose. EvalGlass prevents it: a sink takes immutable typed data and produces a string, so it can never claim more than the Scorecard holds, and it cannot change a verdict or an exit.

How a report is produced

A sink’s render(scorecard) returns a string; gate_state(scorecard, metric) derives each metric’s display state from typed fields.
The verdict word is read from the payload, never written as a literal — so a report can’t say “pass” when the data says informational.
--format ci emits GitHub annotations from the same typed verdict and status data.

Exit mapping

exit_class_for(scorecard) maps the verdict to an ExitClass, and exit_code to a number: pass / informational → 0, fail / blocked → 1, infrastructure/setup → 2.

JSON first, then renderings scorecard.json and runrecord.json are primary; report.md, the terminal output, and CI annotations are views. Wire CI to the exit code or the JSON — not the Markdown. See Wire CI.

The run directory

Each run writes a directory of artifacts under reports/<run-id>/ — part of the host-owned reports/ tree. The JSON is authoritative; every human-facing report is a rendering of it, and the write is crash-safe: files land first, then a completion marker is stamped last, so a half-written run is detectable rather than silently trusted.

files

reports/<run-id>/
  scorecard.json     # the typed Scorecard — authoritative
  runrecord.json     # the full run evidence trail — authoritative
  report.md          # Markdown rendering
  report.html        # HTML dashboard rendering
  manifest.json      # per-file SHA-256 of the JSON artifacts
  run.complete       # completion marker, written last
  drift.json         # optional: written by `evalglass watch`

Artifact	What it is	Status
`scorecard.json`	The typed `Scorecard` — the authoritative verdict, metrics, authority, and diagnostics.	now
`runrecord.json`	The persisted evidence trail for the whole run.	now
`report.md`	Markdown rendering of the Scorecard.	now
`report.html`	A self-contained HTML dashboard rendering — another view of the same typed source, not a second verdict path.	next
`manifest.json`	Records each JSON artifact’s SHA-256 — the integrity check behind atomic persistence.	experimental
`run.complete`	The completion marker, written last and holding the manifest digest — a partial run never looks complete.	experimental
`drift.json`	A typed drift sidecar written next to the run’s files by watch; a diagnostic, never a verdict.	next

Next steps

The full mapping.

The data it renders.

The CI-annotation sink.