interview_scorecard v1.0.0

name: interview_scorecard description: "Build and run structured hiring interview scorecards: define job-linked competencies, write behaviorally anchored 1 to 5 rating scales, assign questions per interviewer, score independently with evidence notes, and run a panel calibration debrief with bias guardrails. Use when creating a scorecard or rubric for a role, preparing an interview panel, auditing an interview process for consistency or bias, comparing candidates, or running a calibration session."

Interview Scorecard

Purpose

Replace impression-based hiring with a scorecard the whole panel uses the same way: same core questions, same competencies, same anchored scale, independent scores first, evidence in the debrief. A number without an anchor and a quoted behavior is an opinion with formatting. This skill prevents that.

Workflow

1. Start from the job, not the person spec

• Write the role mission in one sentence and 3 to 5 outcomes for the first 6 to 12 months.
• List the duties that produce those outcomes. Rate each duty for importance and frequency.
• Derive 4 to 7 competencies from the top duties. Each competency needs: a plain definition, observable behaviors, and the duty it traces back to. If a competency traces to no duty, cut it.
• Mark any true knockout separately (a legal requirement, a safety certification, a non-negotiable skill floor). Knockouts are pass or fail and are checked once, not scored and averaged away.

2. Build the anchored scale

Use a 1 to 5 scale. Anchor 1, 3, and 5 in writing before any candidate is seen. Anchors 2 and 4 are "between the anchors next to them."

• 5: behavior that predicts clear success on the linked duty, with specifics, ownership, and a result or a sound plan under pressure.
• 3: behavior that predicts the person can do the duty with normal support. Partial, generic, or untested in one important part.
• 1: behavior that predicts failure or harm on the linked duty, or no evidence after a fair probe.

Anchor rules: describe what the candidate said or did, not traits ("seemed sharp" is not an anchor). Use critical incidents from real work on this team when you have them. Keep anchors short enough to scan mid-interview. Full worked examples: references/anchored-rubrics.md.

3. Assign questions, one owner per competency

• Write 1 to 2 core questions per competency: behavioral ("Tell me about a time...") for past evidence, situational ("Here is the situation... what would you do?") for judgment.
• Add 2 allowed probes per question (for example: "What was your part in that?" and "What happened as a result?"). Probes clarify. They do not change the question.
• Assign each competency to one interviewer as owner. Other panelists may note it, only the owner scores it as primary. This stops five people scoring the same thing five ways and nobody scoring the rest.
• Every candidate for the role gets the same core questions in the same order. Job-related follow-ups to clarify an answer are fine and should be noted.

4. Brief the panel before the first interview

• Walk through competencies, anchors, knockouts, and the recommendation bands (step 6).
• State the guardrails in step 7 out loud. Assign a debrief chair who is not the hiring manager when possible.
• Run a calibration round before a new scorecard goes live, or when a panel changes: references/calibration.md.

5. Score independently, with evidence, before any discussion

• Each interviewer submits scores and evidence notes before seeing anyone else's card and before any group chat, thread, or hallway debrief.
• Every score needs an evidence note: a behavior, quote, or work sample fact, plus the question it came from. No evidence note, no valid score.
• Score only your assigned competencies in depth. Give an overall recommendation separately, do not back-fill competency scores from a general impression.

6. Debrief and decide

Run it in this order:

1. Reveal all cards at once.
2. Start with the widest disagreements (a spread of 2 or more points on one competency). The low and high scorers quote their evidence. Nobody argues from vibe, seniority, or urgency to fill the role.
3. Check knockouts. A failed knockout ends the discussion unless the knockout itself was misapplied.
4. Compute the weighted score with bin/scorecard.py (weights set before interviews, summing to 100). Treat the number as an input, not the decision.
5. Record the recommendation band: Strong Hire, Hire, No Hire, Strong No Hire. Record dissent and its evidence in the decision note.

Suggested starting bands, to set in advance and keep fixed for the requisition: weighted average 4.2 and up with no competency below 3 supports Strong Hire; 3.4 to 4.19 supports Hire; below 3.4 supports No Hire; any score of 1 on a core competency backed by evidence supports Strong No Hire review. Adjust bands per role, but never mid-requisition.

7. Bias guardrails (non-negotiable)

• No scoring on likability, similarity to the interviewer, accent, appearance, school prestige, or "culture fit" as an undefined term. If values matter, name the observable behavior (for example: "gives direct feedback without contempt") and score that.
• No protected-characteristic questions or notes. Keep every question and criterion job-related, consistent with the US Uniform Guidelines on Employee Selection Procedures (29 CFR 1607) and local law. When in doubt, route to HR or counsel, do not improvise.
• First impressions do not get a field. If an interviewer forms one, they park it and score the evidence.
• One strong or weak answer does not carry other competencies (halo and horns effect). Score each competency from its own question's evidence.
• Accommodations change how a candidate can show evidence, never the bar for the competency. Note the accommodation process fact, not a diagnosis.
• Keep cards and notes factual and retained per company policy. Write every note as if the candidate and a reviewer will read it, because they may.

8. Audit the scorecard itself

After each requisition, or every quarter for evergreen roles:

• Inter-rater spread: competencies where panelists routinely differ by 2 or more points need better anchors or a new calibration round.
• Score distribution: if almost everyone lands on 3, the anchors or the questions are too soft.
• Outcome check: compare scores with 90-day performance for hires. A competency that never predicts anything gets rewritten or dropped.
• Adverse impact screening: selection rates by group should be reviewed by qualified staff. The four-fifths rule in the Uniform Guidelines is a screening rule of thumb, not a legal verdict. Escalate concerns, do not self-clear them.

Output Contract

Deliver, in order:

1. Scorecard: role, mission, outcomes, competencies with definitions and duty links, weights, knockout list.
2. Anchored rubric: for each competency, written anchors for 5, 3, and 1, plus its core questions and allowed probes, and the assigned interviewer role.
3. Panel brief: order of interviews, who owns what, submission deadline for independent cards, debrief chair.
4. Completed cards and decision note: per-candidate weighted score (from the tool), recommendation band, evidence for the widest disagreements, dissent, final decision and who made it.
5. Audit notes when auditing: spread, distribution, and outcome findings, with the specific anchor or question to fix.

Use the blank form in assets/scorecard-template.md. Validate and total completed cards with:

bash
python3 ~/workspace/skills/interview-scorecard/bin/scorecard.py card.json
python3 ~/workspace/skills/interview-scorecard/bin/scorecard.py card.json --compare other.json

Operating Rules

1. Anchors, weights, questions, and bands are fixed before the first candidate and do not change mid-requisition. If the role truly changes, close the requisition and start a new scorecard version.
2. No evidence note, no score. No independent submission, no seat in the debrief debate on that competency.
3. Never average away a documented knockout failure or an evidence-backed score of 1 on a core competency. Discuss it explicitly.
4. Never invent validity numbers, legal rules, or company policy. Cite the source you used or label the point as team judgment.
5. Do not store candidate personal data beyond what the employer's system and policy allow. This skill produces hiring documents, not a shadow candidate database.
6. If two of these rules conflict with local law or company policy, local law and policy win. Say so in the output.