formative_assessment_design v1.0.0

name: "formative_assessment_design" description: "Design short checks for understanding tied to one objective: 1 to 3 items where every wrong answer maps to a named misconception with a planned reteach move, plus fast scoring, misconception reading, and a reteach versus move on decision. Triggers when building exit tickets, hinge questions, checks for understanding, diagnostic questions, or formative assessment for any subject or training setting." metadata: { "includeInPrompt": true }

Formative Assessment Design

Purpose

A short check is only worth class time if its results change what happens next. Most quick checks fail that test: they mix three objectives into five items, the wrong answers are random, and the teacher ends up with a score and no diagnosis. This skill builds checks that read like instruments. One objective, one to three items, every wrong option built from a known wrong rule, and a decision written down before the check is given. It works for a classroom lesson, a tutoring session, a training module, or a self-check inside a course. For the research behind the method, see references/evidence.md. For complete checks in math, science, language arts, and history, see references/worked-check-sets.md. For wrong rules to build distractors from, see references/misconception-catalog.md.

Workflow

1. Write the objective as a performance. One objective per check, stated as what the learner will do, under what conditions, without what support. Weak: "fractions". Strong: "compare two fractions with different denominators by reasoning about size, without converting first". If the sentence names two skills joined by "and", split it and check the first one. Everything else in the check hangs off this sentence.

1. List the wrong rules before writing any item. For this objective, name two to four ways a learner can be wrong in a way that teaches you something: the wrong rule they are following, in plain words. Pull from references/misconception-catalog.md, from errors in recent work, or from a probe tradition in your subject (science has a long published line of these). A wrong rule sounds like "larger denominator means larger fraction". "Careless" and "did not study" are not wrong rules and build nothing.

1. Build one item per wrong rule cluster, three items at most. Each item tests the objective one way, has one correct option, and gives every other option to a named wrong rule from step 2. Scaffold and score with the bundled tool:

sh
bin/check.py new --out fractions-check.json
bin/check.py validate fractions-check.json

Validation fails on any distractor with no misconception named or no reteach move planned. That failure is the point. An unmapped wrong answer is information thrown away. Keep each item answerable in about two minutes or less, in plain language, with no trick wording. Learners should miss items for conceptual reasons, never for decoding reasons.

1. Add a second tier when a right answer could hide a wrong rule. Some learners pick the correct option with broken reasoning, and a single tier scores them as ready. Add one line: "Explain in one sentence why you chose that answer." Keeley's science probes work exactly this way, choice first, explanation second. Read tier two for the correct picks first. A correct choice with a wrong-rule explanation gets scored as that misconception, and the learner joins that reteach group.

1. Write the decision rules before giving the check. Decide in advance, and write it in the check plan:

• 80 percent or more correct on an item: proceed. Note the few who missed it for individual follow-up.
• 50 to 79 percent: brief reteach aimed at the dominant misconception only, then a fresh recheck item in the same session or the next one.
• Below 50 percent: stop and reteach the objective with a different representation, not the same explanation said louder. Recheck before building new material on top of it.
• Any single misconception picked by at least 3 learners, or by 20 percent of the group: that group gets targeted work even when the class as a whole proceeds. These bands are a starting rule of thumb, not a law. Adjust for hard items, for guessing on four-option items, and for objectives that the next topic depends on heavily. What is not adjustable: the decision gets made from the pattern, in the open, right after the check.

1. Collect from everyone, at the same time, without a grade. Simultaneous written or device responses, or an exit ticket collected from every learner. Hands up and "does everyone get it" sample the confident and teach the rest to stay quiet. Say plainly that the check is not graded and what it decides: "This tells me whether we move on tomorrow or fix one thing first." Grading a formative check trains learners to hide gaps, and hidden gaps are the one result this tool cannot use.

1. Score by pattern, not by total. Sort responses into correct, then one pile per misconception. The bundled tool does this from a simple CSV with anonymous learner codes:

sh
bin/check.py analyze fractions-check.json responses.csv

The analyzer prints the per item correct rate, the decision band, each misconception count, and the reteach move you planned for every flagged group. Fast sorting matters: a check you cannot read in a few minutes gets read never, and the lesson moves on blind. If no single misconception stands out, read the explanation lines before reteaching anything. Scattered errors usually point at the item or the teaching, not at one wrong rule.

1. Act on the pattern, then recheck with a fresh item. Reteach means teaching against the named wrong rule: a contrast pair where the wrong rule gives a tempting wrong answer and then fails in the open, a different representation (bar model instead of symbols, timeline instead of paragraph), or a worked example that names the trap. Then give a new item on the same objective. Same surface is fine, same item is not: repeating the item measures memory of the item. Record the result next to the first one. Two patterns worth naming in the record: fixed (recheck clean) and sticky (same wrong rule after reteach, needs a different approach or individual work, not a third round of the same).

1. Give feedback that names the gap and the next move. Short, tied to the task, no grade and no praise aimed at the person. Shape: what the objective asked for, where this response stands against it, the one move that closes the gap. "You compared the numerators only. Compare the size of the parts too: fifths are bigger than eighths, so with the same count, 3/5 wins. Try the next one that way." Feedback at the task and process level does the work. Comments about the learner do not.

Output Contract

• A check spec (JSON from bin/check.py) with one objective, 1 to 3 items, a correct option per item, and for every distractor a named misconception plus a planned reteach move. The spec must pass bin/check.py validate.
• A decision sheet written before the check is given: the proceed, brief reteach, and substantial reteach bands, and the targeted group rule.
• A scoring summary after the check: per item correct rate, misconception counts, flagged groups with learner codes (never real names in files shared or stored outside the class system), and the decision taken.
• A recheck item, written fresh, for every objective that went to reteach, plus the recheck result recorded as fixed or sticky.
• Learner facing feedback for flagged learners: gap, next move, one practice item. No grades, no rankings.

Operating Rules

1. One objective per check. A check that covers three objectives diagnoses none of them.
2. Every wrong answer must earn its place. If a distractor maps to no known wrong rule, replace it with one that does, or cut the option.
3. No planned response, no check. If nothing will change based on the results, save the class time and teach instead.
4. Never grade formative checks, and say so before learners answer. The signal is the product. Grades tax it.
5. Collect from everyone at once. Volunteer answers and confidence polls are not evidence about the class.
6. Read results by misconception pile, never by average alone. A 70 percent average can be one fixable wrong rule shared by most of the class, or five separate problems. The response differs, so the reading must.
7. Reteach against the rule, not around it. Repeating the original explanation to a learner following a different rule adds volume, not understanding.
8. Recheck with a fresh item and record fixed or sticky. An unrecorded reteach will be repeated blind next term.
9. Keep learner data anonymous outside the class system. Use codes in CSVs, spreadsheets, and any tool that leaves the room. Class lists with real names stay in the system your school or organization already approved for them.
10. Do not promise gains. The research supports formative assessment as a practice across many studies. It does not guarantee a result for one class, one learner, or one check, and this skill never quotes an effect size as a promise.
11. Watch the item before blaming the learner. When strong learners miss an item in the same way, or the explanation lines show the wording misled people, fix the item and say so. A bad item read as a misconception poisons the reteach that follows.