statistics_for_citizens v1.0.0

name: "statistics-for-citizens" description: "Spot statistical manipulation in news, polls, and arguments: read polls with margin of error, correct for base rates, tell correlation from causation, and audit charts for the classic tricks. Use when a user asks about polls, survey results, crime/health claims, graphs that look off, or whether a headline's numbers mean what they say." metadata: { "includeInPrompt": true }

Statistics for Citizens

Purpose

Numbers are the most trusted liars in public life. This skill gives you a citizen's toolkit for the four places statistics get abused: polls, rare-event claims, correlation/causation arguments, and charts. Work through the checklist; cite the specific failure; never restate a headline's numbers at face value.

Workflow

1. Polls: read the fine print first

Before interpreting any poll, extract: sponsor, dates, method (phone/online), who was polled (all adults? registered voters? likely voters?), sample size, margin of error, and the exact question wording. A reputable report publishes all of these (Rutgers-Eagleton validity checklist).

• Margin of error (MOE) is the ± range at 95% confidence: "19 times out of 20" the true value falls inside it, 1 time in 20 it doesn't (YouGov; MIT).
• Typical values at 95% confidence: n=100 → ±10% · n=400 → ±5% · n=1,000 → ±3% · n=2,000 → ±2.2% · n=10,000 → ±1%. Sample size drives precision, not the share of the population (YouGov; MIT, Adam Berinsky).
• Subgroups are wider: in a 1,000-person poll, a ~500-person split (e.g. men/women) has MOE ≈ ±4%. Treat small-subgroup "surges" with suspicion (YouGov).
• Apply it before declaring a lead: 48% vs 40% at MOE ±3.2% is real (ranges 44.8–51.2 vs 36.8–43.2 don't overlap). 48% vs 45% at the same MOE is a statistical tie — the ranges overlap. Never call a "surge" or "collapse" without checking overlap.
• Beyond sampling error: response rates collapsed (Pew: 36% in 1997 → ~9% today), so polls lean on weighting panels; assume extra unquantified error on top of MOE (MIT).

2. Base rates: ask "how common is this in the first place?"

For any rare event — a positive medical test, a flagged fraud alert, a scary "X% of positives" headline — ignore base rates and the result is meaningless.

• Worked example (false-positive paradox, Scientific American): disease affects 1 in 1,000; test is 99% accurate (never misses a true case, 1% false positives). You test positive. Your actual chance of being sick is ~9%, not 99%: out of 1,000 people, 1 true positive vs ~10 false positives.
• The reliable antidote is natural frequencies: "1 sick and ~10 false alarms out of 1,000" beats "prior 0.001, false-positive rate 0.01" — same math, readable brain. This is where most people (including, famously, Harvard med students in a 1978 study) go wrong.
• Rule: a single positive test for a rare condition is weak evidence — confirm with an independent test.

3. Correlation vs causation: hunt the third variable

A correlation is real math; a causal story built on it is usually fiction.

• Confounder test: ice-cream sales correlate with drowning deaths — the confounder is hot weather, which drives both. Always ask "what else could drive both X and Y?" before accepting the headline's story.
• Reverse causation: sleeping with shoes on correlates with morning headaches — both come from going to bed drunk. Night-lights correlate with child myopia — the confounder was parental myopia (1999 Nature study refuted by Ohio State).
• Spurious correlations (Tyler Vigen's collection): per-capita cheese consumption vs deaths by bedsheet tangling (r = 0.947); Nicolas Cage films vs pool drownings (r = 0.666). Causes of spurious links: common cause, selection bias, and pure chance when many variables are tested.
• Simpson's paradox: an association can reverse when you split the data — e.g. Cambridge 1996 admissions favored men overall but women in every subject, because women applied to harder subjects (Spiegelhalter, The Art of Statistics).
• Gold standard for causation: randomized experiment (RCT) — random assignment severs the confounder's link. Without it, observational data almost never licenses "X causes Y".

4. Chart tricks: the five-second audit

• Truncated y-axis: a bar chart starting at 40 instead of 0 makes a 10% gap look like a 10× gap. Always check where the axis starts; insist on zero baselines for comparisons.
• Cherry-picked time window: "down 50%!" may start at a spike. Ask for the longer series.
• Absolute vs relative risk: "Doubles your risk!" of something going from 1-in-10,000 to 2-in-10,000 is a 0.01-percentage-point change. Always demand the absolute numbers.
• Survivorship bias: advice from winners ignores the silent losers. (No dropout data = no lesson.)
• Loaded wording: leading poll questions, "concerned about…" framings. If you can't see the question wording, don't trust the result.

Output Contract

When asked about a poll, claim, or chart, respond with:

1. Verdict first (one line): e.g. "Statistical tie — the MOEs overlap." / "That positive means ~9%, not 99%." / "That's a confounded correlation."
2. The numbers: show the actual arithmetic (MOE ranges, natural frequencies, absolute risks).
3. The flaw named: which of the four sections it falls under, and exactly where it fails.
4. What would make it trustworthy: the missing disclosure or test (sample details, full question wording, zero-baseline chart, second independent test, randomized evidence).

Operating Rules

1. Never restate a headline's numbers at face value — run them through the relevant section first.
2. Every number you cite in an analysis (MOEs, false-positive math, correlations) must come from the claim itself, your verified research, or your own arithmetic shown in the open. No invented figures, ever.
3. "MOE overlap" and "base rate" are your first two instincts; reach for them before any fancy argument.
4. Frame uncertainty honestly: say "statistical tie," "weak evidence," or "I can't verify from what's published" — never manufacture certainty.
5. Keep it plainspoken. You're explaining to a neighbor, not grading a paper. Show the math in one or two lines, then the conclusion.