Local-first
It runs on your machine. Your repo stays the source of truth.
EvalGlass is local-first: the vendored runtime runs on your machine and in CI, your repo stays the source of truth, and nothing is sent to a hosted EvalGlass service.
EvalGlass runs entirely on your machine and in your CI. There is no hosted tier, no keys to buy, and nothing phones home. Local-first isn’t a feature — it’s the design.
The three honest “no”s
on your machine & repo
- datasets, traces & prompts
- the vendored runtime (
evals/_evalglass/) - scorecards & RunRecords
- thresholds, calibration & authority
never leaves it
- no telemetry — nothing phones home
- no provider keys on the run path
- no hosted platform or dashboard
- No platform to pay for. There is no hosted EvalGlass, no dashboard service, no SaaS. Evaluation happens in your repo, against your evidence.
- No API keys to buy. The required path needs no provider SDKs and no model keys. Live models are an optional, opt-in lane you own — not a toll on the core.
- No telemetry leaving your repo. EvalGlass collects nothing and sends nothing. Your datasets, traces, and scorecards stay where you put them.
Why local-first matters
There are no servers to keep running on the project’s behalf. A tool that runs entirely from vendored source in your repo does not switch off when a bill goes unpaid — it keeps working as long as your machine and your CI do.
What runs where — who owns the truth
- EvalGlass owns the effect-free Evaluation Core, the runtime harness, the installer, and the vendored runtime under
evals/_evalglass/— plain code you can read. - You own your datasets, rubrics, thresholds, calibration evidence, baselines, host-evaluator code, and your data policy. The agent populates host-owned files; it never fills in authority.
- Nothing is silently overwritten. Re-vendoring replaces the managed runtime; your host-owned truth is left untouched.
The runtime outlives the tooling.
Because authority lives in host-owned files and the managed runtime is plain vendored code, the evaluation keeps working after the plugin and any coding-agent context are removed. Local-first means there is no cost centre to monetise and no place to insert a meter — EvalGlass measures what your evidence supports and reports only what it can honestly claim. Host ownership, in detail →
Related