interview_moderation_review v1.0.0

---

name: interview_moderation_review description: Review your own moderation from a research interview transcript: measure talk ratio, turns, and interruptions, code every moderator question for leading, loaded, and double-barreled faults, log the moments a participant answer was steered, and turn the repeating pattern into two or three habits to change before the next session. Use after a user research, qualitative, or discovery interview when a transcript or recording exists and the moderator wants an honest self-review or a peer review, not a summary of findings.

---

Interview Moderation Review

Purpose

Find out how much of an interview's data is the participant talking, and how much is the moderator steering. A scorecard that counts talk time and interruptions is not enough, because the expensive faults are in the wording: a question that assumes a feeling, restates the participant in stronger words, or hands them a benefit to agree with. Those faults bend the answers that follow, and the findings built on those answers inherit the bend.

This skill reviews the moderator, not the participant and not the findings. It produces counts you can compare across sessions, a log of the exact moments an answer was steered with the quote that proves it, and a short change plan built from the faults that repeat. One bad question is a slip. The same fault in three sessions is a habit, and habits are what the plan targets.

Workflow

1. Prepare the transcript

• Work from a verbatim transcript with speaker labels and one block per turn. If the transcript merges speakers or cleans up half-finished sentences, go back to the recording for the turns you code, because cut-offs and restarts are evidence.

• Number the turns before coding. Turn numbers are the address system for the whole review. Every claim in the output points to one.

• Mark what counts as talk before counting. State in the header whether introductions, consent, and tech setup are in or out, and use the same rule for every session you compare. Word counts are the default unit. Use timed talk share instead only if the transcript carries reliable timestamps, and say which unit you used.

• Keep names out. Refer to the participant by their study code in the coding sheet, the log, and the plan.

2. Measure talk balance and interruptions

• Run the bundled script for the first pass counts:

python3 ~/workspace/skills/interview-moderation-review/scripts/moderation_metrics.py <transcript.txt>

Add --json when another tool needs the numbers. The script reports word counts and talk share by speaker, turns and average turn length, moderator question turns, the longest moderator turn, interruptions with explicit transcript evidence, and candidate flags for human coding.

• Read the talk share against the rule of thumb from NN/g and dscout: aim for the participant to speak for roughly 80 percent of the interview, with the moderator near 20 percent and that share mainly questions. Treat 80 percent as a flag, not a grade. A session at 65 percent with clean questions can be sound, and a session at 82 percent can still be steered by three bad questions.

• Read the longest moderator turn out loud. A long turn is usually a preamble that explains the design, a story of the moderator's own, or several questions stacked into one breath. Code it on the sheet in the next step.

• Count interruptions from evidence, not vibes. In text, the evidence is a participant turn that ends unfinished, a transcriber overlap marker, or a participant who restarts or drops a point in the next turn. The script counts overlap markers and participant turns ending in -- before a moderator turn. If your transcript has neither, check the audio for every moderator turn that follows a long participant turn, and write "interruptions not countable from this transcript" rather than reporting zero as if it were a finding.

• Separate a deliberate redirect from a cut-in. A redirect happens at a sentence boundary, gives a plain reason such as time, and returns the floor. A cut-in takes the floor mid-point. They get different codes and different fixes.

3. Code every moderator turn

• Fill one row per moderator turn on the turn-coding sheet in references/turn-coding-sheet.md, using the definitions in references/coding-definitions.md. Quote the moderator's exact words in the evidence column.

• Code the question faults first: LEAD-SUGGEST (an answer suggested in the question), LEAD-ASSUME (a feeling or fact assumed that the participant never stated), LEAD-REPHRASE (the participant restated in the moderator's stronger words), LEAD-NAME (an internal element or term named before the participant used it), LOADED (a premise the participant must accept to answer at all), DOUBLE (two subjects, one answer), CLOSED (a yes or no frame doing the job of a main question), and TAG (a statement fishing for agreement). NN/g defines a leading question as one that includes or implies the desired answer in its phrasing, and lists rephrasing in your own words, suggesting an answer, naming an interface element, and assuming a feeling as the common traps.

• Then code the floor faults: LONG and MONOLOGUE-OPEN turns, ACK-EVAL praise such as "great" or "perfect" where a neutral "okay" or "mmhmm" belongs, SELF-DISCLOSE stories told to relate, and CUT-IN or REDIRECT moves.

• Code the participant turn that follows each moderator turn with an effect code: MIMIC when the participant repeats the moderator's wording or suggested benefit, CORRECT when they spend the turn fixing the premise, SHRINK when the answer collapses to a short agreement after fuller earlier answers, SPLIT when they have to split a double question into two verdicts, and EXPAND when they add unprompted detail. Effects are how you tell a fault that bent data from a fault a strong participant pushed through.

• Do not skip turns that look fine. Neutral questions and the EXPAND turns after them are the control case. They show which of the moderator's moves earn detail, and the change plan must protect those moves.

4. Log the steering moments

• Move each confirmed moment into the steering-moment log in references/steering-moment-log.md. Confirmed means a coded fault followed by MIMIC, CORRECT, SHRINK, or SPLIT, or a cut-in that cost the participant a point.

• Write the loss in one sentence per moment. Name what the study can no longer claim from that exchange, for example that a need appearing in the findings came from the moderator's question, not the participant.

• Write a neutral rewrite for every logged moment. Use the participant's own words, offer both poles where a feeling belongs ("What was easy or difficult about...?"), and cut any preamble about the design or the team. Rewrites are practice material for the next session, not decoration.

• Flag any finding already in synthesis that rests on a MIMIC or CORRECT exchange. Tell whoever owns the synthesis, in plain terms, which claim is bent and which turn bent it. Do not quietly fix the transcript or the notes.

5. Score the pattern across sessions

• Review the last three interviews before judging a habit, following the self-assessment practice described by dscout: listen to or read the full session, evaluate against a fixed template, then pick a small number of points to work on. One session shows slips. Three sessions show the pattern, and they protect you from rebuilding your style around one hard participant.

• Tally codes and pattern tags across the three sessions. Rank faults by how often they appear times how much data they bent. A rare fault with a MIMIC effect on a core topic outranks a frequent mild one on small talk.

• Note the conditions, not just the counts. Faults that cluster late in the session, on one topic, or with one participant type point to a cause worth naming, such as rushing the last guide section or defending a design the moderator worked on.

• If a peer can review one transcript, take it. Swap transcripts, code independently, and compare the steering log before discussing. Agreement on the moments matters more than agreement on every code.

6. Turn the pattern into a change plan

• Pick two or three habits at most, from the top of the ranked pattern. The dscout practice picks two points to work on. A short list you will actually run in the next session beats a full report card.

• Write each habit as trigger, replacement move, and check. The trigger is the moment you can catch in the room, such as starting a question with "You mentioned..." The replacement is a move you can do without thinking, such as an echo: repeat the participant's exact last words with a questioning tone. NN/g's echo, boomerang, and Columbo techniques are the standard replacements. Echo repeats their words, boomerang returns their question with "What do you think?", and Columbo asks part of a question and trails off so the participant completes it.

• Add the floor habits that fit your pattern. Counting to three in your head after a participant finishes gives them room to continue, per dscout. Counting to ten before deciding whether to speak at all is NN/g's rule for usability tests. Neutral acknowledgments ("okay", "mmhmm") replace praise. A written opening of a few plain lines, with the study purpose kept vague, replaces the design explanation.

• Set the check before the next session starts: which codes must fall, which effect codes must disappear, and what talk share you expect. Write the target next to the habit. "Fewer leading questions" is not a check.

• Re-review after the next three interviews and compare tallies side by side. Keep a habit on the plan until its codes stay down across three sessions, then promote the next fault from the ranking.

Output Contract

Deliver these five pieces, in this order:

1. Header counts: participant and moderator talk share with the unit and inclusion rule stated, turn counts, average turn lengths, moderator question turns, longest moderator turn quoted, and the interruption count with its evidence basis (or a plain statement that interruptions are not countable from this transcript).
2. The completed turn-coding sheet for the session, one row per moderator turn, with exact quotes and next-turn effect codes.
3. The steering-moment log: one entry per confirmed moment, with the moderator quote, the participant response, the loss in one sentence, a neutral rewrite, and a pattern tag.
4. The three-session pattern summary: code tallies across the last three reviews, the ranked faults, and any condition the faults cluster under. If only one session exists, say so and label every pattern claim as single-session.
5. The change plan: two or three habits, each written as trigger, replacement move, and check with a numeric or zero target, plus the date or session count for the re-review.

Also deliver, when it applies, a short bent-findings note naming any synthesis claim that rests on a steered exchange, with the turn numbers.

A worked example of all five pieces on a short flawed transcript is in references/worked-review.md, with its source transcript in references/sample-flawed-transcript.txt.

Operating Rules

• Judge the words in the transcript, not the moderator's intent, experience, or reputation. Most steering is unintentional. Intent explains a fault, it does not clear it.

• Never present the script's candidate flags as verdicts. The script matches patterns. A human coder who has read the definitions decides, and the coding sheet is the record of that decision.

• Never invent a count. If a number did not come from the script, a timestamp, or a manual tally you can show, leave it out. Talk share quoted to a decimal from a transcript with no word count is a fabricated number.

• Treat roughly 80 percent participant talk as a rule of thumb from NN/g and dscout, not a validated threshold. Report the number, name the benchmark, and let the coding carry the judgment.

• Quote exactly or not at all. A paraphrased quote in a steering log hides the very rephrasing fault the review exists to catch.

• Keep the review on the moderator's own work. Do not use it to grade a participant, and do not share a named moderator's log outside the agreed review circle without their consent.

• Protect participant privacy in every output. Study codes only, no names, no identifying details in quotes, and retain the log no longer than the study data it belongs to.

• Style rule for all outputs of this skill: plain sentences, no em dashes, no filler praise about the moderator or the participant. The review is blunt because the next participant deserves a cleaner interview.

References

• references/coding-definitions.md for the code list with NN/g and survey research examples and neutral rewrites.
• references/turn-coding-sheet.md for the per-session coding sheet and tally block.
• references/steering-moment-log.md for the log entry format and log rules.
• references/worked-review.md and references/sample-flawed-transcript.txt for a complete review with real script counts.
• scripts/moderation_metrics.py for talk balance, turn stats, explicit interruption evidence, and candidate flags.
• Nielsen Norman Group, Avoid Leading Questions: https://www.nngroup.com/articles/leading-questions/
• Nielsen Norman Group, 5 Facilitation Mistakes to Avoid During User Interviews: https://www.nngroup.com/articles/interview-facilitation-mistakes/
• Nielsen Norman Group, User Interviews 101: https://www.nngroup.com/articles/user-interviews/
• Nielsen Norman Group, Talking with Users in a Usability Test: https://www.nngroup.com/articles/talking-to-users/
• dscout, 6 Common Mistakes to Avoid During Qualitative Interviews: https://www.dscout.com/people-nerds/qualitative-interview-mistakes