Frequently asked questions

EvalGlass is an AI quality-control tool that lets your coding agent build and run project-specific checks for your AI application. It studies your repo, traces, prompts, and model calls; runs checks locally and in CI; and returns bounded scorecards. See: Overview

No. There is deliberately no certify, approve, or gate verb. A scorecard is a bounded claim about a measured run — it never asserts your AI application is correct, safe, or production-ready. See: No false green

It means EvalGlass measured behavior but no active, host-approved gate ran — so the result is evidence, not enforcement. It exits 0, but exit 0 does not mean quality was enforced. See: Verdicts & exit codes

No. EvalGlass is local-first: the vendored runtime runs on your machine and in your CI, your repo stays the source of truth, there is no telemetry, and no provider keys are required on the run path. See: Local-first

Importing exported traces ships now — local JSONL, OpenTelemetry, and OpenInference exports. A live provider pull (Langfuse, Phoenix, LangSmith) is an experimental, opt-in lane reachable with the connect --live verb — credentials are env-var names, data_policy defaults to unknown, no provider SDK is imported, and a pull is proposed and cannot gate. See: Connectors

Not yet. EvalGlass is v0.2.1 pre-alpha, tagged, and not yet published to PyPI. It is open source under Apache-2.0 and usable today, but contract stability is still settling. See: Roadmap & status

Claude Code is the primary plugin and launch surface. Codex is supported as a second coding-agent runtime through the same evaluation model; its public marketplace listing is a separate next step. See: Codex runtime

You promote it deliberately: validate a dataset, approve a threshold, set the metric to gating, and calibrate a judge if one is used. Authority comes from host-owned records, never from the plugin or the agent. See: Promote a gate

EvalGlass is a product family. Core is the open-source, local product you install today — it runs the evaluations you define into bounded scorecards you can inspect, compare and keep. Discovery reads a bounded representation of your application and proposes the evaluations it is missing — you review, edit and accept, and accepted proposals become portable Core artifacts. Intelligence explains why an evaluation failed — causal diagnosis with an explicit evidence level, a verified repair, and an evaluation learning loop — and may include Discovery for covered applications. Discovery and Intelligence are separate commercial products, in private preview. See: What EvalGlass is (and isn't)

Yes. Core is Apache-2.0, local-first, and free — installing or running it needs no account and no payment. The commercial products fund the harder work around it; you never pay for permission to use Core. See: License

No. Installing the open-source plugin installs Core only. Discovery and Intelligence are separate, opt-in products and are never silently enabled — adopting either is a deliberate step that changes nothing about Core's license or its local independence. See: Local-first

No standing combined access, by design. You choose the application and the evidence window; access is scoped to what you invoke, with separate least-privilege identities for the repo, trace, and model adapters. Intelligence is in private preview. See: Security

No. Intelligence explains why an evaluation failed through causal diagnosis, and can stand on its own; for covered applications an Intelligence engagement may include Discovery so proposed and accepted evaluations feed the same investigation. Either way, the Core-runnable artifacts they produce remain usable whether or not you adopt the other. Both are in private preview. See: Sustainability

Yes. Release or organizational assurance is a use of EvalGlass evidence, not a separate product — Discovery and Intelligence produce technical artifacts (scorecards, evaluation candidates, root-cause records, repair verifications) that you can export into your existing GRC, audit, observability, and CI systems. EvalGlass is not your system of record; requirements from regulation or internal controls are inputs that can guide Discovery. Discovery and Intelligence are in private preview. See: Security

You do. Open artifacts a paid product generates — scenarios, regressions, rubrics — remain customer-controlled and keep working in Core after any engagement ends. The product that creates evidence never owns your decision or your evidence. See: License

Not yet. Both are in private preview — not generally available. We state where each stands plainly, and never show either executing or producing a live customer result. See: Honesty charter