--- name: results_to_claims_mapping description: "Mapping results to claims: list every claim a paper or report makes, tie each claim to the specific result inside the same document that supports it, flag claims with no support or with support that is weaker, narrower, or different from the claim, and rewrite each flagged claim to match the evidence. Ships a claim ledger, a tested ledger checker and claim scanner, ten support checks, and worked examples of claims cut down to size. Use when drafting or reviewing an abstract, discussion, conclusion, executive summary, release post, or report whose conclusions need to match its results, when a claim sounds stronger than the data, or before a paper or report ships. For checking external citations use citation_integrity_audit, for full manuscript review use peer_review_assistant." --- # Results to Claims Mapping ## Purpose Keep a document honest with itself. A citation audit checks whether outside sources back a claim. This skill checks something different and more basic: whether a paper or report's own results back its own claims, in the same document, at the strength the claim uses. The failure is common and measured. Boutron and colleagues defined spin as reporting strategies, whatever their motive, that highlight a benefit despite a nonsignificant primary outcome or distract from nonsignificant results. In their sample of 72 trial reports with nonsignificant primary outcomes, 42 (58.3 percent) had spin in the abstract conclusions. The CONSORT guidance for abstracts states the plain rule this skill enforces: conclusions must be consistent with the results, balance benefits and harms, and avoid over-generalisation. The output is a claim ledger: every load-bearing claim, the result that carries it, a verdict, and a rewrite for every claim that fails. The document does not ship with OVERSTATED, UNSUPPORTED, or CONTRADICTED claims still standing in their original wording. ## Workflow ### 1. Build the result register first List what the document actually found before reading what it claims. One row per result, numbered R1, R2, and so on: - What was measured, in whom, over what period. - The value for each group or condition, the effect size, and its precision (confidence interval where reported). - Whether the analysis was prespecified or exploratory, primary or secondary, and where it sits: table, figure, or results paragraph. - Null results and harms get rows too. A result nobody claims is still part of the evidence, and step 6 needs it. If the document reports a prespecified plan, registry entry, or analysis plan, note which results match it. The COMPare project built its whole method on that comparison, registry against published report, and found outcome discrepancies in most of the trials it assessed. A result that appears in the report but not the plan is exploratory by default, whatever the text calls it. ### 2. Extract the claims Go section by section: title, abstract, summary, discussion, conclusion, recommendations, and any headline or release copy shipped with the document. A statement is a claim for the ledger if it: - Asserts a finding, an effect, a comparison, or a cause. - Gives or implies a number, a direction, or a size. - Recommends an action on the basis of the findings. - Characterises the whole study ("the first to show", "safe and effective", "ready for use"). Copy the claim verbatim. Do not tidy it before judging it, because the tidying is often where the overstatement hides. Start the ledger from `assets/claim-ledger.csv`, or generate one: ``` python3 scripts/claim_map.py init ledger.csv python3 scripts/claim_map.py scan draft.md ``` The scanner prints candidate sentences (strong verbs, causal verbs, numbers, recommendation wording). Candidates only: decide each one by hand, and add claims the scanner missed, especially claims carried by a heading or a figure caption. Split compound claims. "The program works, is safe, and should be rolled out" is three claims with three different evidence needs. ### 3. Map each claim to its results For each claim, name the result IDs that are supposed to carry it and the locator for each: table, figure, or results paragraph. Then state the warrant in one sentence: why does this result justify this claim? Use Toulmin's split, claim, data, warrant, qualifier. Most failures surface at this step, because the warrant has to be said out loud: - No result named: the claim is UNSUPPORTED until one is found. - The warrant needs a fact the study did not measure (a mechanism, a longer timeframe, a different population): the claim outruns its data. - The warrant works only with a qualifier the claim does not carry (one site, four weeks, exploratory): the claim is OVERSTATED and the qualifier goes into the rewrite. ### 4. Run the ten support checks Run every mapped claim through all ten checks in `references/support-checks.md`: outcome identity, direction, magnitude, statistical status, population and setting, timeframe, design and causality, certainty and qualifiers, completeness, and claim type separation. Record which check failed, not just that something feels off. The named spin patterns at the end of that file (misleading reporting, misleading interpretation, inappropriate extrapolation, following Yavchitz and colleagues) are the fast scan for the common shapes. Two checks catch the most damage: - Statistical status. A nonsignificant primary outcome cannot be carried by a significant secondary one. A nonsignificant result also cannot be claimed as proof of no effect or of equivalence. Both upgrades fail. - Design and causality. Observational, survey, and cross-sectional results carry association verbs. Causal verbs, and causal nouns such as "the effect of X on Y", need an experimental design or an explicit argument for why the leap is earned. ### 5. Give a verdict per claim - SUPPORTED: every check passes at the claim's stated strength. - QUALIFIED: the result supports the claim once its scope, population, timeframe, or certainty is stated. The rewrite adds the qualifier. - OVERSTATED: a real result sits underneath, but the claim is stronger, broader, more certain, or more causal than the result. - UNSUPPORTED: no result in the document carries the claim. - CONTRADICTED: a result in the document points the other way, or a result the claim depends on omitting contradicts it. - NOT_A_RESULT: interpretation, speculation, or background that makes no claim on this document's data. Allowed, if it is written as what it is and not dressed in result language. Validate the ledger before rewriting: ``` python3 scripts/claim_map.py check ledger.csv python3 scripts/claim_map.py summary ledger.csv ``` The checker fails the run if a SUPPORTED or QUALIFIED claim names no result, or a failed verdict has no rewrite. It warns on causal language attached to a non-experimental design and on results named without a locator. ### 6. Check the orphans in both directions Claims with no result are half the audit. Results with no claim are the other half. List results that no claim mentions and ask why: - A null primary outcome with no claim is a reporting failure, not a relief. The abstract and conclusion still have to carry it. - A harm with no claim has to appear wherever the benefit claim appears. - A result that was prespecified and never reported anywhere in the document gets named in the report as missing, with the reason if the document gives one. ### 7. Rewrite each flagged claim to match the evidence Rewrite rules, in order: 1. Keep the true part. Most overstated claims have a supported claim inside them. Find it and keep its numbers exact. 2. Lead with the primary result. If the headline claim rests on a secondary or exploratory result, the rewrite states the primary result first. 3. Downgrade the verb to the design: causes becomes was associated with, proves becomes suggests, eliminates becomes reduced in this sample. 4. Add the missing scope: population, setting, timeframe, and the comparison actually made. 5. Restore what was omitted: the null result, the harm, the group where the result reversed, the count of outcomes tested. 6. Move recommendations out of result language. A recommendation states the result, then the judgement, then what would change the judgement. 7. If nothing in the results can carry any version of the claim, cut the claim. A cut is a valid outcome and gets recorded as one. Worked rewrites for a null primary trial abstract, an observational churn report, a benchmark release post, and one claim that passes untouched are in `references/worked-examples.md`. ### 8. Final consistency sweep - Read the title, abstract, and conclusion against the ledger, not against memory. Every claim there must have a SUPPORTED or QUALIFIED verdict in its rewritten form. - Check every number in a claim against the result register, digit for digit. - Check that the limitations section and the claims agree. A limitation that would sink a claim if it were true cannot sit quietly at the back while the claim runs unqualified at the front. - Re-run the checker. The ledger ships with the report. ## Output Contract Deliver, in this order: 1. The result register: numbered results with locators, including nulls and harms. 2. The claim ledger (CSV plus a readable rendering): claim verbatim, section, result IDs and locators, design, verdict, and the rewrite for every claim that is not SUPPORTED. 3. The orphan list: results no claim mentions, with a line on each saying whether it must enter the text or can stay in the results section, and why. 4. A short summary: counts by verdict, the three worst failures with the check each one failed, and the cuts made. 5. The corrected text for the title, abstract, and conclusion, or a statement that they already match the results. ## Operating Rules - Judge the claim as written, in the section where it appears. A claim qualified in the discussion is still unqualified in the abstract, and most readers only meet the abstract. - Never upgrade a failed check into a pass because the claim matters to the authors. Importance is a reason to gather more evidence, not to borrow strength the results do not have. - Never invent a supporting result. If the locator cannot be named, the verdict cannot be SUPPORTED. - Keep prevalence claims tied to their samples. The spin figures in this skill come from biomedical trials with nonsignificant primary outcomes. They show the failure is real and common in that setting. They do not measure any other field, and the skill text must not present them as if they do. - A mapping that flags every claim is as broken as one that flags none. Record clean passes as SUPPORTED and leave the wording alone. - Speculation is allowed when it is labelled as speculation. Recommendations are allowed when they are argued, not smuggled in as findings. - Do not soften a verdict in the summary that the ledger states plainly. The summary and the ledger tell the same story.