eval-driven-development v1.0.0

name: eval-driven-development description: Build quality into LLM apps with evals before you build the app itself. Sets up curated golden datasets, calibrated LLM-as-judge scoring, and regression gates in CI. Use when shipping an LLM feature, an agent, a RAG pipeline, or a chatbot; when a prompt or model change might have broken behavior; or when a repo needs an eval harness a coding agent can drive.

Eval-Driven Development

Purpose

EDD is the top-down way to build LLM apps: define what "good" looks like as evals first, then build toward them. It borrows the spirit of test-driven development, but treating it exactly like TDD burns your API budget and teaches the wrong lessons. Evals are expensive, slow, and partly subjective, so the discipline is: few, high-quality cases, metrics that actually matter, and a gate that blocks regressions. The loop is three steps:

1. Curate ~100 goldens with expected outputs.
2. Define 3-5 metrics tied to what good looks like.
3. Iterate until every metric passes on your test cases.

Workflow

1. Define success. Ask what good looks like for real users and write 3-5 metrics tied to that, not vanity metrics. Add security metrics as non-negotiable: prompt injection, PII leakage, jailbreak refusal. If the agent has a retrieval step or tools, add component-level unit evals (context precision/recall, tool selection accuracy) alongside the end-to-end regression tests.
2. Build the golden dataset. Run bin/edd.py init <task> to scaffold eval/<task>/. Start with ~100 curated cases, scale to 500 at most. One JSON object per line: input, expected reference answer, deterministic assertions (must_contain, must_not_contain, json_keys). Version the dataset in git like code. Never auto-generate thousands of cases to pad coverage; mass-generated sets rot into noise.
3. Deterministic checks first. Cheap assertions (keyword coverage, JSON schema, forbidden terms) run before any LLM judging. They are fast, free, and stable. Redteam/security cases gate at 100 percent: one complied jailbreak fails the build.
4. Add LLM-as-judge for the soft dimensions (faithfulness, helpfulness, on-task). Write an explicit rubric with concrete criteria, explicit fail examples, and a reference answer wherever possible. Pin the judge model and version-lock the rubric in git. Have the judge reason first, then emit a structured verdict (JSON), so every judgment can be parsed and audited.
5. Calibrate the judge against humans. Human-label 15-20 samples, run the judge on the same set, and compute agreement with bin/edd.py calibrate labels.csv. Ship at 80 percent agreement or better and Cohen's kappa of 0.6 or better; rewrite the rubric below 0.4. Freeze the rubric version, and re-calibrate after every rubric edit or judge model change, since a rubric tweak can silently shift judgments elsewhere.
6. Run the loop. Generate outputs, score them, inspect the failures, change the prompt/model/logic, re-run. When a coding agent drives this, the eval harness is what it measures against; without it there is no way to tell whether the agent it wrote actually behaves. Ship the version that survived the evals, not the first version that looked fine.
7. Gate CI. Keep thresholds per category in thresholds.yaml. Compare every run against a saved baseline report and exit non-zero on any regression. Use a deterministic mock provider so CI runs without API keys. Report pass rates per category plus score distributions, and use confidence intervals before celebrating small gaps (a 2 percent move on 100 samples is noise).

Output Contract

• eval/<task>/goldens.jsonl, a versioned judge rubric, and thresholds.yaml
• Per-run report: pass rates by category, with per-case failure diagnostics
• CI gate verdict: pass/fail, with every failing category named
• Calibration record: label count, agreement rate, kappa, rubric version

Operating Rules

1. The judge must be a different model family than the generator. A model that judges its own output grades itself.
2. Name the biases in the judge prompt: position, verbosity, self-preference, style-over-substance. Counter them: swap-and-average candidate order for pairwise comparisons; use pairwise for model-vs-model picks and pointwise scoring for CI gates; penalize verbose padding explicitly in the rubric.
3. Reference-guided grading beats open judging: give the judge a gold answer and ask whether the candidate matches it.
4. Never optimize blindly toward the judge's quirks. Re-validate against the human-labeled set on a schedule; an uncalibrated judge is not a measurement.
5. Small and curated beats large and auto-generated, every time.
6. Redteam gates are absolute, never averaged away.
7. Accuracy: every factual claim in evals and reports must come from a web search or a primary source. Unverifiable claims get cut, not shipped.