Use case · Agentic app evaluation
Does your tool-using agent do the right thing?
Evaluating an agentic app means checking behavior across steps — tool selection, trajectories, and outcomes. EvalGlass turns traces you have already exported into project-specific checks your coding agent runs locally, and returns a bounded scorecard you can gate in CI when you choose.
Evaluate my agent's tool use and trajectories against my expected checks.
non_evaluable because its trajectory was empty — never scored as 0.0.
Verdict informational: an aggregate green here still would not prove the agent succeeded.
What it checks
Tool selection · step trajectories · task outcome · grounding and policy adherence —
scoped to the unit you declare: a traces: route can set
unit: trajectory|session|step next to grade
the whole agent run, not one call. Checks are authored around your AI application’s real
behavior, not a packaged agent benchmark.
Grade the whole run next
A single call is not the agent. A traces: route can declare
unit: trajectory|session|step to grade the whole agent run, wiring the
trajectory_shape@1 aggregate. Egress is worst-of-members (fail-closed) —
the run is only as trustworthy as its weakest graded member. It is informational and cannot
gate, and an empty trajectory is non_evaluable, never
0.0. An aggregate green is still not proof the agent succeeded.
The scorecard
unit: trajectory0.78 worst-of-membersVerdict informational — graded at the trajectory unit;
the empty run is non_evaluable, never 0.0, and nothing gates until you
approve a threshold. Illustrative example, not a measured result.
Measured, not certified
EvalGlass measures what you defined and reports what it can claim. It never certifies the agent is safe or correct; authority to gate comes from host-owned records, and a fresh setup gates nothing.
What it will not claim
An aggregate green is not proof the agent succeeded. A whole-run grade rolls up the members it could measure — and a right answer can still come from the wrong tool or a skipped step. EvalGlass grades the observable trace evidence, never the model’s self-narrated reasoning, and nothing gates until you approve a threshold in your own config. No false green →
Ask your coding agent.
Evaluate my agentic app using EvalGlass.