Contracts Reference

Enums

Every closed value set, in one place.

These are the fixed enumerations the contracts use. They are closed sets — a value outside them fails parsing rather than being silently coerced. Other pages link here instead of redefining values.

Measurement

ScoreStatus

ValueMeaning
scoredA meaningful value was produced.
blockedA required metric could not honestly run.
non_evaluableThe example lacks evidence for this metric.
skippedThe metric was intentionally not run.
errorThe evaluator or evidence parser failed unexpectedly.

Validity

ValueMeaning
validThe value is a sound measurement.
invalidA value was produced but cannot be trusted.
not_measuredNo measurement was attempted.
not_applicableThe metric does not apply to this example.

Severity

ValueMeaning
info · warning · errorDiagnostic severity — describes the finding, not the run outcome.

Metrics

EnumValues
Lensreference, non_reference
ScoreTypebinary, continuous
Directionhigher_is_better, lower_is_better
Aggregationmean, median, min, max, rate, none

Units & evidence

EnumValues
UnitKindcall, step, trajectory, session
DataPolicypermitted, redacted, forbidden, missing, unknown
JudgeEvidenceStatusok, timeout, provider_error, malformed, missing (only ok may carry a value)
TraceFormatlocal, opentelemetry, openinference

Authority

EnumValues
DatasetStatusproposed, validated, retired
MetricStatusdraft, informational, calibrating, gating, retired
ThresholdApprovalproposed, approved
JudgeCalibrationuncalibrated, calibrating, calibrated, drifted, retired
AuthorityLevelnone, informational, gating

Outcome

EnumValues
Verdictinformational, pass, fail, blocked
BaselineStatecomparable, not_comparable, missing_baseline, comparison_not_requested
ExitClasszero (0), nonzero_fail (1), nonzero_blocked (1), infrastructure_error (2)

Extension lanes

EnumValues
LaneStatusran, skipped, blocked (never a score or verdict)
LanePorttrace_source, score_sink, judge_model, task_runner

Confidence & decisions

EnumValues
IntervalMethodwilson (proportions), student_t (means); none is a rejected sentinel. See Estimate.
DecisionStatisticpoint, lower_confidence_bound, upper_confidence_bound; null resolves to the conservative bound for the direction. See Decision Policy.
GrantStatusmatched, mismatched, expired, missing. See Judge Integrity.

Next steps