affinity_mapping v1.0.0

name: affinity_mapping description: >- Cluster raw user research observations into findings a team can defend: code interview and usability notes against a written codebook, group codes bottom-up into themes, turn topic labels into finding statements, flag themes that rest on one participant, run a coder agreement pilot where the approach calls for one, and keep the quote, participant and session behind every claim. Use when synthesizing interviews, usability sessions, field notes or open-ended survey answers, when a readout needs an audit trail, or when themes feel tidy but thin. Triggers on "affinity mapping", "affinity diagram", "synthesize these interviews", "cluster these notes", "thematic analysis", "codebook", "themes from this research".

Affinity Mapping

Purpose

Turn a pile of interview notes, usability observations and open-ended answers into findings that hold up when someone asks "says who, in which session, in what words". Affinity diagramming, devised by Jiro Kawakita in the 1960s as the KJ method, groups individual observations by their natural relationships instead of sorting them into categories chosen in advance. This skill adds the parts a quick workshop version usually skips: a written codebook before clustering, a check on how much weight each theme can carry, and a readout where every claim traces back to a quote.

Use it for applied product and UX research synthesis. For a formal academic thematic analysis, follow the approach rules in step 1 and cite the method you actually used.

Workflow

1. Pick the approach before touching the data, and write it down

Choose one, and record the choice at the top of the codebook:

• Codebook approach: codes are defined in a written codebook, partly in advance and partly from the data. Fits team research, applied questions and repeat studies.
• Coding-reliability approach: a fixed codebook plus a measured agreement pilot between coders. Fits work where a second team must be able to reproduce the coding.
• Reflexive thematic analysis (Braun and Clarke): coding stays open and interpretive, themes are built by the researcher, and subjectivity is part of the method. In this approach there is no fixed codebook and no agreement score. The quality check is a documented discussion of other readings of the same data.

Mixing these up is the common failure. Do not report a kappa score for a reflexive study, and do not skip the reliability pilot for a coding-reliability study and call the result rigorous. Braun and Clarke's six phases still describe the shape of the work in any approach: get familiar with the data, generate codes, build candidate themes, review themes, define and name themes, write the report. The phases loop back on each other. They are not a one-way checklist.

2. Make one observation per note, with its source attached

Break transcripts and session notes into single-idea observations. One note carries one observation, one participant id, one session id, and a verbatim quote or a close observation note. A note that bundles three ideas will fit three clusters badly.

Keep paraphrase out of the quote field. If you shorten a quote, mark the cut. A reader who cannot see the original wording cannot judge the claim built on it.

Lay the observations out in a table with columns observation_id, participant_id, session_id, quote, code and theme. The example in assets/example-observations.csv shows the shape, and scripts/theme_check.py reads exactly that shape.

3. Build the codebook, then code against it

Start from references/codebook-template.md. Every code gets a definition, an inclusion rule, an exclusion rule that names the near miss and where it belongs instead, and one real positive and negative example with participant and session ids.

Code the whole set against the current codebook version. When a segment fits two codes, apply both and note the overlap instead of forcing a single choice. When a segment fits no code, park it in an unclassified row and review the parked set before clustering. New codes usually hide there.

Change the codebook only at a version boundary. Record the change, then recode the earlier segments the change affects, or write down why they stand.

4. Run the agreement pilot when the approach calls for one

For codebook and coding-reliability work, have a second coder code a pilot subset independently, before seeing your codes. Pick the subset on purpose, covering early and late sessions and at least one difficult transcript, and record how you picked it.

Score it with:

python3 scripts/theme_check.py --agreement agreement.csv

The CSV needs unit_id, coder_a and coder_b. The script reports percent agreement (Po), chance agreement (Pe) and Cohen kappa, computed as (Po - Pe) / (1 - Pe), plus every disagreement by unit. Kappa sits at or below percent agreement for the same data, because it removes the agreement chance alone would produce. Read kappa with the commonly used Landis and Koch style bands: 0.00 to 0.20 slight, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, 0.81 to 1.00 almost perfect. Those bands are a convention, not a law. Other published sets, such as McHugh's, are stricter. Always report Po next to kappa, because when one code dominates the data kappa can read low even when coders agree often.

Treat disagreements as codebook defects first and coder errors second. Each disagreement should end in a split code, a merged code, a sharper exclusion rule, or a recorded boundary call. Never settle a disagreement by vote and leave the codebook unchanged. Cohen kappa covers two coders. For three or more, use a multi-rater measure such as Fleiss kappa or Krippendorff alpha, and say which one you used.

For reflexive work, skip the pilot. Instead, have a second reader argue for a different reading of three themes, and file the argument and your response with the worksheet.

5. Cluster bottom-up, by content before labels

Group observations and codes by what they share, following the NN/g order: generate the notes, organize them into clusters, then prioritize clusters and next steps. Practical rules from that method and from the KJ tradition:

• Sort first, name later. A label chosen early pulls borderline notes toward it.
• Let clusters form from the notes, not from the research questions, the org chart, or last quarter's themes.
• Do not force a cluster. A note that fits nowhere stays unclustered until its group shows up, and a small cluster is still a cluster.
• When a note genuinely belongs in two clusters, copy it into both and mark the copy, rather than arguing it into one home.
• Split any cluster whose one-sentence summary needs "and" to join two different ideas.
• Keep outliers visible in their own corner of the board. An outlier is often the first sign of a segment the sample barely reached.

6. Check every theme before it becomes a finding

Fill in references/clustering-worksheet.md for each cluster, then run:

python3 scripts/theme_check.py observations.csv

The script flags themes resting on a single participant, themes with a single observation, duplicate observation ids, and observations missing a quote, session, code or theme. Then do the checks a script cannot do: internal fit, separation from neighbouring themes, a search for counter-evidence, and the worst-fit test from the worksheet.

Handle a single-participant theme honestly. A theme does not need to appear in every interview to matter, and a rare theme can be the most important one in the set. But one participant cannot support a claim about participants in general. Demote it to a lead, attach the follow-up question that would test it, and label it as a lead in the readout.

7. Rewrite topic labels as finding statements

A cluster label names a topic, such as "reports" or "onboarding". A finding says what was learned, for whom, and under what condition, such as "when the standard report cannot produce the cut a manager asks for, participants move the work into a spreadsheet rather than wait". If your sentence would still be true before the research ran, it is a topic, not a finding. Rewrite it.

Attach to each finding its observation ids, participant count, session count, strongest verbatim quote, known counter-evidence or limit, and a confidence call: pattern, lead, or contradiction. references/worked-synthesis.md walks a seven-observation set through every step, including a flagged theme that gets demoted in the readout.

8. Prioritize and hand over the trail

Close by ranking findings by the decision they inform, not by cluster size alone. A large cluster can be low stakes, and a small one can block a launch. Save the coded observations table, the codebook version used, the filled worksheet and the board state with the readout. Synthesis that cannot be re-walked by someone else is an opinion with good formatting.

Output Contract

• A coded observations table: one row per observation, with participant id, session id, verbatim quote, code and theme, in the CSV shape scripts/theme_check.py reads.
• The codebook used, with version and change log, built from references/codebook-template.md.
• A filled clustering and theme-check worksheet per study, from references/clustering-worksheet.md, including the script output.
• For codebook and coding-reliability work, the agreement pilot result: units double-coded, Po, kappa, and the codebook change each disagreement produced. For reflexive work, the alternative-reading note instead.
• A readout of finding statements, each with evidence counts, a verbatim quote with participant and session ids, counter-evidence or limits, and a pattern, lead or contradiction label.
• A leads list: every single-participant or single-session theme, named as a lead, with the follow-up that would settle it.

Operating Rules

1. Every factual claim in a skill or a readout must be verified via web search or a primary source. Never invent numbers, dates, quotes, or events. Unverifiable means cut it or frame it as opinion.
2. No claim without a trail. If a finding sentence cannot be pointed back to observation ids and quotes in under a minute, the trail gets fixed before the readout ships.
3. Frequency is not importance, and importance is not proof. Report participant and session counts for every theme, and let the reader see both the size and the weight you gave it.
4. Never average away a contradiction. When participants disagree, report both sides with their counts, instead of writing the majority view as the finding.
5. Keep participant wording intact. Quotes are verbatim, cuts are marked, and no quote gets cleaned up into words the participant did not say.
6. Do not name themes in the plural voice of all users when the evidence covers a few. Match the claim's scope to the sample that supports it.
7. Protect participants. Use ids, not names, in the observations table, the worksheet and the readout, and keep any key linking ids to people outside the synthesis files.