feature_parity_audit v1.0.0

name: "feature_parity_audit" description: "Score a rebuild, migration, or replacement against the product it replaces: inventory every feature, classify it must, should, could, or wont with MoSCoW, test each row in both versions, weight gaps by user impact, and gate the switch with a scoring script that returns SHIP, SHIP WITH WAIVERS, or NO-SHIP. Use when moving to a new platform, vendor, codebase, or redesign, when someone claims the new version does everything the old one did, or before a cutover, launch, or contract switch."

Feature Parity Audit

Purpose

Stop a replacement from shipping on opinion. Build one matrix that lists what the old product does, how much each item matters, and what testing in both versions actually showed, then let a fixed gate decide whether the switch is ready.

This is a standalone audit for any migration or redesign. It is not a migration plan and it does not move data. It answers one question with evidence: what will users lose if we switch now?

Workflow

1. Name the pair and the switch date. Write down the old version, the new version, and what switching means in practice: cutover, forced upgrade, contract end, or redirect. An audit with no switch event drifts into a wish list.
2. Inventory from evidence, not memory. List features from the old product itself: menus, settings, admin screens, API docs, usage logs, support tickets, and analytics. Sit with one real user workflow at a time, from start to finish. A feature is a completed workflow, not a button. If nobody can show the feature working in the old version, it does not get a row.
3. Classify each row with MoSCoW. Must: the switch fails without it, legally, functionally, or because a core daily workflow breaks. Should: important, painful to lose, but users can work around it for a short time. Could: a real improvement whose absence nobody feels in week one. Wont: agreed out of scope this time, written down so the decision stays visible. MoSCoW comes from Dai Clegg in 1994 and the DSDM tradition. Its discipline is a small Must set. If more than about 60 percent of the rows are Must, the classification has failed: reclassify before testing.
4. Set impact per row, 1 to 5. Impact is how badly users feel the gap, not how hard the feature is to build. 5 means a daily core workflow breaks for many users. 1 means a rarely used extra disappears. Use usage data where it exists. Old products carry features built for workflows that no longer exist, and usage logs usually show a much smaller load bearing set than the full inventory.
5. Freeze the gate before testing. The default gate is in scripts/parity_score.py: any Must row that is not a clean pass blocks the switch, and Should parity below 85 percent needs a written waiver. A team may change the thresholds, but only before the first test run, and the change goes in the audit record. Changing the gate after seeing the score is not a decision, it is a rewrite.
6. Test every row in both versions. Use the same account type, the same data, and the same steps on each side. Record evidence in both columns: what was run, when, and what happened. Pick the method per row from references/testing-parity.md: side by side workflow tests by default, ported business logic tests for core rules, characterization tests when nobody knows what the old system really does, mirrored runs for services and data jobs.
7. Score honestly. Status values are pass, partial, fail, and untested. Partial means the workflow finishes only with a workaround or a reduced result, and the workaround goes in the notes. Untested scores zero. Never mark a row pass from a demo, a roadmap promise, or a vendor claim.
8. Run the scorer. python3 scripts/parity_score.py assets/parity-matrix-template.csv on your filled copy, or add --json for machine readable output. Exit codes: 0 SHIP, 1 SHIP WITH WAIVERS, 2 NO-SHIP, 3 bad input. Read the blocker and waiver lists, not just the headline percentage. A high average can hide one broken core workflow, as the worked example shows.
9. Act on the verdict. NO-SHIP: fix the blockers or reclassify a row with a written reason, then retest only the changed rows and rerun the script. SHIP WITH WAIVERS: each waiver gets a named owner, a fix date, and a plain note to affected users about what is missing and for how long. SHIP: keep the matrix as the launch record.
10. Close the audit after the switch. Two to four weeks later, compare member or customer complaints against the matrix. A complaint about a row marked pass is a testing failure. A complaint about a row never listed is an inventory failure. Record which one it was and fix that step in the next audit.

Output Contract

• A completed parity matrix CSV in the format of assets/parity-matrix-template.csv, one row per workflow, with priority, status, impact, and evidence for both versions
• A scorer report showing per tier parity, the blocker list, the waiver list, and the verdict
• For every waiver: owner, fix date, and the user facing note that will be sent
• For every deliberate difference from the old product: one line in the notes column saying the difference is on purpose and why
• A short closing note after the switch that classifies any parity complaints as testing failures or inventory failures

Operating Rules

1. Never gate a switch on a percentage alone. One failing Must row outweighs a good average.
2. Never test with only a fresh admin account. Restricted accounts, old data, and error states are where parity breaks.
3. Never let a roadmap promise, a demo, or a vendor statement count as a pass. Only a run task in both versions counts.
4. Never reclassify a Must to Should after it fails, without a written reason signed off by the person who owns the user impact.
5. Never change gate thresholds after testing starts. Record any change made before testing in the audit file.
6. Wont rows stay in the matrix. Deleting them hides a decision and invites the argument again at launch.
7. Keep feature parity and data parity separate. A workflow can pass while its records, totals, or history are wrong. Score data checks as their own Must rows when the data matters.
8. If the new product is better in a different way, record that as a gain in the notes. Gains do not cancel a failing Must row. They get weighed by the decision maker, outside the score.