Runtime & Integrations · Runtime Harness
Run Config
From YAML to typed, authority-safe config.
evalglass.yaml loads into a typed RuntimeConfig with
yaml.safe_load. It fails closed on malformed input and uses
authority-safe defaults: an under-specified metric stays
informational + proposed and
cannot gate.
Config is the boundary where a host states intent — and the place a mistake could silently create a gate. So the loader is conservative: it converts external shapes immediately, rejects invalid ones, and never lets the file grant authority it cannot back.
What it rejects
- An out-of-range threshold, or a duplicate metric name.
- A judge metric with no calibration — forced to
UNCALIBRATED. - Anything malformed: a config error becomes a setup diagnostic, not an evaluator score.
It reuses the core MetricSpec.from_dict, so a metric’s declared meaning is validated
the same way whether it comes from config or code. The sub-configs —
Dataset, Trace, Task, Judge,
Rubric, MetricConfig — are plain typed dataclasses with no I/O.
The config cannot self-approve
Writing
threshold_approval: approved is intent, not authority. Approval lives in
host-owned records the config references; an under-specified metric is defaulted back to a
non-gating state. See Promote a Gate.