debugging_playbook v1.0.0

name: debugging-playbook description: "Systematic debugging: reproduce, isolate, hypothesize, fix, verify. Use when diagnosing a bug, bisecting a regression, shrinking a failing test case, or when a fix keeps recurring."

Debugging Playbook

Purpose

Replace guess-and-patch with a repeatable loop: reproduce → isolate → hypothesize → fix → verify. The method is older than the tooling — Andreas Zeller formalized delta debugging in 1999 ("Yesterday, my program worked. Today, it does not. Why?", ESEC/FSE '99), and Toyota's Five Whys (Sakichi Toyoda, 1930s; formalized by Taiichi Ohno in Toyota Production System, 1988) supplies the root-cause discipline. The goal is always the root cause, not symptom suppression.

Workflow

1. REPRODUCE — no repro, no diagnosis

• Get the bug to happen on demand: exact steps, inputs, environment.
• Capture the exact error output and stack trace before touching code.
• Flaky? Characterize the non-determinism first: shared state between tests, time-dependent assertions, unbounded concurrency, external dependencies.
• If you cannot reproduce locally, identify the environmental difference (OS, dependency version, config, data volume) — or instrument the failure site to capture state on the next occurrence.

2. ISOLATE — shrink the surface until only the bug remains

Pick the right narrowing tool for the bug type:

• Regression from a known-good state → git bisect.

bash
  git bisect start
  git bisect bad                    # current HEAD is broken
  git bisect good v1.2.0            # last known good
  # at each midpoint: run the repro, then:
  git bisect good   # or: git bisect bad
  git bisect reset                  # when done
  

• Bloated failing input → delta debugging (bin/minimize.py). Shrinks a failing file to the smallest set of lines that still reproduces the failure (1-minimal: removing any single remaining line kills the repro).

bash
  python3 bin/minimize.py failing_case.py \
      --cmd "python3 {file}" --marker "ValueError: boom"
  

--cmd runs with {file} replaced by each candidate; interesting means the marker appears in combined output (or use --exit 42 to match an exit code). Full algorithm detail: references/ddmin.md.

• No VCS, no minimizer → halve the code surface manually. Comment out or stub half the suspect code; if the bug survives, it's in the remaining half.
• Write each suspect boundary down — "works above this line, breaks below it."

3. HYPOTHESIZE — state it in writing before testing it

• Generate 2–3 hypotheses, ranked by likelihood. Writing them down exposes gaps in what you think you know.
• For each hypothesis, design the cheapest experiment that would disprove it (a log line, a targeted print, a one-off script) — falsification beats confirmation.
• Test one hypothesis at a time. Never change several things at once.
• Recurring bug? Run the Five Whys once: ask "why?" of each answer until the cause is systemic and actionable (five is a heuristic, not a rule). Stop at a cause a person or team can actually change. "Operator error" is never a root cause — ask one more why.

4. FIX — minimal change at the root cause

• One guard in a shared function beats a guard at every caller.
• Fix the cause your evidence pointed to, not the symptom.
• Keep the change small; resist the urge to refactor while fixing.

5. VERIFY — prove it and prevent recurrence

• Repro no longer reproduces; full test suite still green.
• Add a regression test that would have caught this failure mode.
• Remove all debug code, prints, and breakpoints before committing.

Output Contract

Every debugging engagement ends with a report containing:

1. Root cause — what specifically caused it (one paragraph).
2. Evidence — the stack trace, log line, or test that proves it.
3. Fix — the change that resolves it (diff or file:line reference).
4. Prevention — the regression test or safeguard added.

Operating Rules

• Never guess. A fix without a confirmed root cause is a patch on a ghost.
• Reproduce first, fix last. No reproduction step means no verified fix.
• One hypothesis, one change, one test at a time. Parallel changes make causality unknowable.
• Evidence at every step. Keep the failing output, the bisect log, the minimized case, and the hypothesis list — they are the deliverable's receipts.
• Treat the crash as a symptom, not the disease. Read stack traces top-down, reason bottom-up.
• Prefer observation over interruption. Add logging before reaching for a debugger when the bug is timing-sensitive; a debugger can mask races.