---
name: eval-driven-development
description: Build quality into LLM apps with evals before you build the app itself. Sets up curated golden datasets, calibrated LLM-as-judge scoring, and regression gates in CI. Use when shipping an LLM feature, an agent, a RAG pipeline, or a chatbot; when a prompt or model change might have broken behavior; or when a repo needs an eval harness a coding agent can drive.
---

# Eval-Driven Development

## Purpose

EDD is the top-down way to build LLM apps: define what "good" looks like as
evals first, then build toward them. It borrows the spirit of test-driven
development, but treating it exactly like TDD burns your API budget and teaches
the wrong lessons. Evals are expensive, slow, and partly subjective, so the
discipline is: few, high-quality cases, metrics that actually matter, and a
gate that blocks regressions. The loop is three steps:

1. Curate ~100 goldens with expected outputs.
2. Define 3-5 metrics tied to what good looks like.
3. Iterate until every metric passes on your test cases.

## Workflow

1. **Define success.** Ask what good looks like for real users and write 3-5
   metrics tied to that, not vanity metrics. Add security metrics as
   non-negotiable: prompt injection, PII leakage, jailbreak refusal. If the
   agent has a retrieval step or tools, add component-level unit evals
   (context precision/recall, tool selection accuracy) alongside the
   end-to-end regression tests.
2. **Build the golden dataset.** Run `bin/edd.py init <task>` to scaffold
   `eval/<task>/`. Start with ~100 curated cases, scale to 500 at most. One
   JSON object per line: input, expected reference answer, deterministic
   assertions (`must_contain`, `must_not_contain`, `json_keys`). Version the
   dataset in git like code. Never auto-generate thousands of cases to pad
   coverage; mass-generated sets rot into noise.
3. **Deterministic checks first.** Cheap assertions (keyword coverage, JSON
   schema, forbidden terms) run before any LLM judging. They are fast, free,
   and stable. Redteam/security cases gate at 100 percent: one complied
   jailbreak fails the build.
4. **Add LLM-as-judge for the soft dimensions** (faithfulness, helpfulness,
   on-task). Write an explicit rubric with concrete criteria, explicit fail
   examples, and a reference answer wherever possible. Pin the judge model
   and version-lock the rubric in git. Have the judge reason first, then emit
   a structured verdict (JSON), so every judgment can be parsed and audited.
5. **Calibrate the judge against humans.** Human-label 15-20 samples, run the
   judge on the same set, and compute agreement with
   `bin/edd.py calibrate labels.csv`. Ship at 80 percent agreement or better
   and Cohen's kappa of 0.6 or better; rewrite the rubric below 0.4. Freeze
   the rubric version, and re-calibrate after every rubric edit or judge
   model change, since a rubric tweak can silently shift judgments elsewhere.
6. **Run the loop.** Generate outputs, score them, inspect the failures,
   change the prompt/model/logic, re-run. When a coding agent drives this,
   the eval harness is what it measures against; without it there is no way
   to tell whether the agent it wrote actually behaves. Ship the version that
   survived the evals, not the first version that looked fine.
7. **Gate CI.** Keep thresholds per category in `thresholds.yaml`. Compare
   every run against a saved baseline report and exit non-zero on any
   regression. Use a deterministic mock provider so CI runs without API keys.
   Report pass rates per category plus score distributions, and use
   confidence intervals before celebrating small gaps (a 2 percent move on
   100 samples is noise).

## Output Contract

- `eval/<task>/goldens.jsonl`, a versioned judge rubric, and `thresholds.yaml`
- Per-run report: pass rates by category, with per-case failure diagnostics
- CI gate verdict: pass/fail, with every failing category named
- Calibration record: label count, agreement rate, kappa, rubric version

## Operating Rules

1. The judge must be a different model family than the generator. A model
   that judges its own output grades itself.
2. Name the biases in the judge prompt: position, verbosity, self-preference,
   style-over-substance. Counter them: swap-and-average candidate order for
   pairwise comparisons; use pairwise for model-vs-model picks and pointwise
   scoring for CI gates; penalize verbose padding explicitly in the rubric.
3. Reference-guided grading beats open judging: give the judge a gold answer
   and ask whether the candidate matches it.
4. Never optimize blindly toward the judge's quirks. Re-validate against the
   human-labeled set on a schedule; an uncalibrated judge is not a
   measurement.
5. Small and curated beats large and auto-generated, every time.
6. Redteam gates are absolute, never averaged away.
7. Accuracy: every factual claim in evals and reports must come from a web
   search or a primary source. Unverifiable claims get cut, not shipped.