Evaluations fitted to your system, owned as code, open source.

Install

Two commands, then ask.

/plugin marketplace add Evalglass/evalglass-core
/plugin install evalglass-core@evalglass

“Core” is a product name only. The commands, the EvalGlass/evalglass-core repository, the Python package, the Claude Code plugin and the scorecard schemas all stay named evalglass — nothing is renamed for marketing. Installing Core does not enrol your repo in any hosted service. All the ways in →

Core’s job

Ask, Observe, Shape, Run, Read, Improve.

Core compiles explicit contracts into runnable checks: your agent reads what is actually there — schemas, prompts, traces, known failure patterns — and turns them into evaluations you own. The loop your agent runs:

The artifact

A scorecard — Core’s primary proof.

Core’s boundary

Core executes what you define.

Core is the evaluation execution substrate. You author metrics, scenarios, datasets, rubrics and thresholds — or select generic starter packs — and Core runs, compares and retains them. It automates the mechanical work of setup and authoring, but it deliberately does not inspect your application to infer what should be evaluated:

• Call-site scaffolding. now A conservative, read-only scan finds candidate LLM call sites and surfaces data-policy boundaries at setup — installation support only. It never imports or runs your code.
• Explicit authoring & starter packs. now Author your own metrics and rubrics, or select generic, user-chosen starter packs. A starter pack is not application-specific coverage — selecting one does not mean your application is evaluated.
• User-directed generation. planned An opt-in generator runs a generation spec you author (“paraphrase these approved examples”); it never infers a missing risk or designs the objective from your application. Every generated case lands proposed, never validated reference data.

Governance is invariant: a scenario, rubric or judge is a proposal. A rubric does not make its judge calibrated; a judge produces evidence, not authority, until you calibrate it. Deciding what your application should be tested for — inspecting it to propose the evaluation agenda — is a different job: that is EvalGlass Discovery.

The complete runtime — minus discovery

Core keeps the whole epistemic system. It is not a stripped installer.

Narrowing Core removes the reasoning that infers what to test — nothing else. The full evaluation and epistemic runtime stays open source and stays in Core:

• The epistemic core. The Verdict Engine and honest non-scored states (blocked, not_measured, incomparable), confidence intervals and lower-bound gating, comparability, provenance and immutable run records.
• Built-in metrics & judges. A library of built-in metrics and instruments, and every judge adapter (fake, command, OpenAI-compatible) with the calibration harness — a metric library is not discovery.
• Reports, baselines & drift. Markdown and HTML reports, scorecard rendering, baselines and release deltas, and drift on the series you choose to monitor.
• Traces, CI & connectors. Explicit trace import (OpenTelemetry, OpenInference, Langfuse, Phoenix, LangSmith), CI wiring and result sinks — evidence adapters, not discovery engines.

Core can even load and run a metric catalog Discovery produced, without a Discovery licence — it just never generates one from your application. Everything that executes, compares, calibrates, transports or preserves an explicit evaluation stays here.

Core does not

A rigorous laboratory — not the one who chooses the experiments.

To keep the open boundary honest, Core deliberately excludes:

• Application-specific discovery. It does not inspect a repository to infer what should be evaluated, reconstruct latent workflows, or discover application-specific failure modes.
• Automatic synthesis from context. It does not automatically propose metrics, scenarios or rubrics from your code, prompts or traces, or recommend an evaluator from an inferred task.
• Causal diagnosis and learning. It does not diagnose cause, attribute root cause, plan interventions, or maintain an autonomous evaluation-learning loop.
• Organizational governance. It does not manage policy, control, applicability or portfolio state, and it issues no legal or compliance conclusions.

Core gives the team a rigorous laboratory. Discovery chooses the experiments. Intelligence explains the hard results.

CI and host authority

Core proposes; the host approves the gate.

A fresh or example run is informational, never a false green. It does not fail your build until you promote a metric to a gate on a threshold you approve. Core measures and reports; you decide and gate. It reads what your AI actually did; it never edits your app, and it never claims the AI is safe.

Rights

Apache-2.0, and the artifacts are yours.

• Apache-2.0. Commercial use, modification and redistribution, with a patent grant and no warranty. Read the licence →
• Customer-owned artifacts. The scorecards, rubrics, calibration records and config your agent generates are yours to keep, edit and run — with or without any commercial product.
• No hosted enrolment. Installing Core does not connect your repo to a service; the runtime is vendored into your repo as plain, readable code.

Core extensions, not commercial products

Optional adapters that cannot create authority.

Core extensions are removable adapters, evaluators, connectors and sinks that participate in Core’s loop. They bring evidence in or send scorecards out — they are not Intelligence, and they can never grant authority.

Browse Core extensions →

What Discovery and Intelligence add

Different jobs — Core stays complete without them.

Core answers what did the checks observe? The other two products answer different questions. They do not make Core incomplete, and they are separate, paid, pre-alpha products.