runbook_quality v1.0.0

name: "runbook_quality" description: "Writes and grades incident runbooks that a tired on-call operator can actually follow at 3am: copy-paste triage commands with expected output, decision trees instead of prose, explicit escalation thresholds, and a quarterly drill cadence. Use when asked to create a runbook, improve or review an existing one, or set up operational readiness for a service."

Runbook Quality

Purpose

Write or grade a runbook against one standard: a person who has never seen this service, running on little sleep, under pressure, must be able to follow it and fix the problem. This skill covers authoring a runbook and scoring an existing one. It does not respond to a specific incident; it makes the response possible.

Workflow

1. Discover. Before writing, get the five basics: which service and environment, the top failure modes it hits, who reads this (team, skill level, timezone coverage), which alerts point at it, and who the escalation contacts are. Ask if any are missing. Never write a runbook for "the service" in the abstract.

1. Set the 3am assumption. The reader knows nothing about this service, is tired, and has minutes before someone senior asks for an update. Every sentence gets judged against that reader. If a step assumes knowledge the reader cannot be counted on to have, spell it out or cut it.

1. Draft from the template. Use the structure in references/runbook-template.md. The load-bearing parts:

• Header tied to the exact alert name, with severity, what user impact looks like, and expected time to resolve.
• Symptom in user-facing terms ("Checkout requests failing"), not infra jargon.
• Time-boxed triage: first 5 minutes, commands in order, each with the exact command, what normal output looks like, and the alert cue that says "this is the problem."
• Decision trees, not paragraphs: labeled branches ("if pods are crashing...") with a trigger condition and distinct steps per branch. Cover the top 3 to 5 root causes, nothing exotic.
• "Make it stop now" before the real fix: rollback, kill switch, load shed. Stopping the bleeding outranks understanding it.
• Escalation stated plainly: when to page the owner, when to declare a major incident, in what order. Include a time trigger (for example, "if not contained in 15 minutes").
• Comms cadence for the severity (SEV1 every 10-15 min, SEV2 every 30 min, SEV3 async): what we know, what we do not know, what we are doing now, next update at.
• Close-out: verify user impact is gone, watch for regression, then update the runbook if anything in it was wrong.

1. Grade it. Score the draft with references/grading-scorecard.md (the five A's: actionable, accessible, accurate, authoritative, adaptable). Then run the tired-operator test: read the whole thing as that reader. Would you find the alert-to-runbook link in under a minute? Is every command copy-pasteable with no editing? Does every branch have a trigger you can actually observe? If a step requires logging in somewhere, the credentials or link are in the runbook. Fix everything that fails before shipping.

1. Schedule the drill. A runbook that has never been tested is a guess. Every runbook gets a quarterly drill: simulate the alert in staging, time a person who did not write it following the steps, and fix what they got stuck on. If the service changed and the runbook did not, the drill will find it. Log the drill date and the fixes in the runbook's changelog.

Output Contract

• A runbook file in markdown following references/runbook-template.md, with the five discovery answers filled in, no bracketed placeholders left.
• A short grading note: the five A's score, what failed the tired-operator test, and what was fixed.
• A drill date: the next scheduled test, named owner, and where the drill will run.

Operating Rules

1. Every factual claim about the service (ports, hostnames, thresholds, contacts) must be verified against the actual environment or a primary source. Guessing a threshold is worse than omitting it; if you cannot verify, mark it as needing the owner to fill in.
2. Commands must run as written. No placeholders inside command text, no "replace X with your value" in the middle of a command block. If a value genuinely varies, show a concrete example and say exactly where the real value comes from.
3. Expected output for every triage command: what normal looks like and the cue that says abnormal. A command without an interpretation is just typing.
4. Branches over prose. If a step is longer than a short paragraph, it probably hides a decision tree. Draw it out.
5. The mitigation comes before the root-cause hunt. Write the "make it stop now" option even when the root cause seems obvious.
6. Never invent alert names, metric values, or contact people. Ask, or leave a clearly-marked fill-in for the service owner.