doc_staleness_detection v1.0.0

name: "doc_staleness_detection" description: "Detect when documentation has drifted from the code: stale-doc signals, automated drift checks, triage, and targeted fixes. Use when someone asks whether docs are accurate, to audit docs before a release, to wire doc-freshness checks into CI, or when a doc example no longer matches the code."

Doc Staleness Detection

Purpose

Find the places where docs and code disagree, rank them by how badly they mislead a reader, and fix the ones that matter. Docs rot in predictable ways: renamed symbols, changed defaults, dead links, API specs nobody re-checks, and examples that stopped working. This skill gives you a repeatable way to catch that drift instead of trusting the docs blindly.

Workflow

1. Scope the audit

Pick the docs that carry risk: README, install guides, API references, architecture docs, runbooks. Skip changelogs and marketing pages; they drift differently and rarely mislead. Write the scope down so the same set gets re-checked next time.

2. Run the automated checks

Use the bundled checker first (bin/check_doc_staleness.py):

bash
python3 bin/check_doc_staleness.py --docs docs/ --src src/ --repo .

It reports, per markdown file:

• SIGNATURE_DRIFT: a Python function/class documented in a fenced code block whose argument list no longer matches the real code.
• MISSING_SYMBOL: a documented name that no longer exists anywhere in the source tree.
• DEAD_LINK: a relative link or #anchor that does not resolve to an existing file or heading.
• STALE_BY_DATE: the code behind a doc section changed in git after the doc was last touched.

Details on the heuristics live in references/signals.md.

For API specs, run the spec-vs-spec and spec-vs-code checks:

bash
oasdiff breaking openapi.base.yaml openapi.current.yaml
spectral lint openapi.current.yaml

See references/signals.md for the CI wiring (GitHub Action recipe with a reviewable override path).

3. Do the semantic pass by hand

Automation catches identifier drift. It does not catch meaning drift. For each doc in scope, extract its atomic claims and check them against current code. The four claim types that rot silently:

1. Behavior flips. The code inverted something the doc describes: sync became async, auth was added or removed, a default toggled. Search by concept keyword (retry, cache, auth, queue), not by symbol name, because the prose shares no tokens with the diff.
2. Counting claims. "The three modes", "these five options". Recount against the enum or config table whenever a variant was added or removed.
3. Negative claims. "X is not supported", "there is no way to Y", "Z is planned". A new feature lands and quietly invalidates these. Grep for them when a capability is added.
4. Version claims. "Requires v2.4" in the doc, pyproject.toml says 2.9. Check every version number in prose against the manifest.

4. Triage the findings

Not every finding deserves a fix. Score each one on two axes: how many readers hit it (README install steps = everyone, internal runbook = few) and how expensive a wrong belief is (wrong auth flow costs days, a dead anchor costs seconds). Fix high-readership or high-cost items in this pass. File the rest as a dated backlog entry with the evidence, so the next audit can re-triage.

5. Update with evidence, then stamp

Rewrite only the stale claim, not the surrounding page. Every updated claim needs a source-of-truth pointer: a file path and line, a spec URL, or a commit. Then stamp the doc: add or bump a Last verified: YYYY-MM-DD header so the next audit knows how old the check is.

Before/after examples:

markdown
# BEFORE (stale)
We use Redux Toolkit with createSlice for all global state.

# AFTER (verified against src/store/, commit 9f2a1c)
We use Zustand for client-side global state (src/store/, verified 2026-10-08).

markdown
# BEFORE (dead flag)
Run the importer with `--batch-mode` to skip prompts.

# AFTER (verified against src/importer/cli.py)
Run the importer with `--non-interactive` to skip prompts
(src/importer/cli.py:44, flag renamed in v1.7).

Output Contract

A staleness audit returns:

• Findings list: one line per finding with type, file:line, the stale text, and the source-of-truth evidence.
• Triage table: finding, readers affected, cost of the wrong belief, fix now or backlog.
• Patch: minimal edits touching only stale claims, each traceable to code.
• Stamp: Last verified: YYYY-MM-DD bumped on every audited doc.

Operating Rules

1. Every claim in the findings must cite evidence: a code location, a spec, a commit, or a test result. Unverifiable = cut it or label it as an opinion, never state it confidently.
2. Do not rewrite healthy pages. Touch only the stale claim; unrelated prose stays.
3. Do not execute repo-local scripts or entry points to verify docs unless the user authored the repo and approved it. Static reading of the parser/route/config source is the default.
4. Distinguish the two directions: doc changed but code never followed (unimplemented spec) is also a finding, and it may mean the code is behind, not the doc. Do not assume the code is always right.
5. Keep tool versions pinned when you put checks in CI. A drift check that silently upgrades itself is a drift source.