rfp_response_evaluation v1.0.0

---

name: rfp_response_evaluation description: "Evaluate RFP responses side by side with a defensible weighted scorecard: pass/fail gates, anchored rubrics, independent scoring with bias controls, total-cost comparison, structured reference checks, and a clean award and debrief record. Use when proposals have arrived and a team must shortlist, score, compare, or defend a vendor selection."

---

RFP Response Evaluation

Purpose

Turn a pile of proposals into a ranked, documented decision that a losing bidder, an auditor, or a finance lead can follow without guessing. The work has five parts: gate out non-compliant bids, score the rest against criteria fixed before proposals arrived, compare total cost on the same horizon, verify claims through references, then record the award rationale and debrief the losers.

Use this skill after an RFP closes. For writing the RFP itself, use a drafting skill first, then return here once responses are in.

Workflow

1. Freeze the rules before opening proposals

Confirm these exist in writing before anyone reads a bid. If any item is missing, create it now and record that it was set before scoring, or stop and flag the gap.

• Criteria list, each one traced to a requirement in the statement of work
• Weight for each criterion, weights summing to 100
• Scoring scale and anchors (see step 3)
• Price scoring method promised in the RFP
• Pass/fail gates (step 2)
• Evaluator roster, conflict-of-interest declarations, and a named evaluation lead
• TCO horizon (commonly 3 or 5 years for software and equipment, 1 to 3 for a short services engagement)

Never add a criterion, change a weight, or change the price method after proposals are opened. That single rule prevents most protests and most internal arguments.

Starter weight sets, adjust to the buy and document why:

| Criterion | Services | Software / SaaS | Construction / capital | |---|---|---|---| | Technical approach and fit | 30% | 30% | 25% | | Experience and qualifications | 20% | 15% | 30% | | Past performance and references | 15% | 15% | 15% | | Price, scored on TCO | 25% | 25% | 25% | | Risk, security, compliance depth | 5% | 10% | 5% | | Implementation and transition plan | 5% | 5% | 0% (fold into approach) | | Total | 100% | 100% | 100% |

2. Run the pass/fail gate first

Check every proposal against mandatory items only, with no scoring:

• Submitted on time, in the required format, with required signatures and forms
• Meets every mandatory requirement stated as must, shall, or required in the RFP
• Holds required licences, certifications, insurance, and eligibility (not debarred, where that applies)
• No material exception to mandatory contract terms

Record pass or fail plus the specific clause for each failure. A failed proposal leaves the pool. Do not score it anyway, and do not let a low price bring it back. Note late or non-responsive bids separately and notify them under the RFP rules.

3. Score independently with an anchored rubric

Default scale, 0 to 5. Write the anchor text into the score sheet so every evaluator uses the same words:

| Score | Anchor | |---|---| | 5 | Exceeds the requirement, with specific evidence in the proposal | | 4 | Fully meets the requirement, with good detail and evidence | | 3 | Meets the requirement adequately, minor gaps that do not affect delivery | | 2 | Partially meets it, material gaps or vague claims | | 1 | Barely addresses it, major gaps | | 0 | Does not address it, or the answer contradicts the requirement |

Rules for this step:

1. Each evaluator scores alone first. No discussion, no shared sheets, no outside research on a bidder during scoring.
2. Score one criterion across all proposals before moving to the next criterion. Randomize proposal order per criterion if the tooling allows it.
3. Score each proposal against the RFP requirement, never against another proposal.
4. Every score needs a note: the page or section, the strength or weakness found, in plain words. A score with no note does not count.
5. Apply the same note to the same strength or weakness wherever it appears. If detail is a strength in one bid, the same detail is a strength in the next.
6. Vendor name and prior familiarity stay out of non-price criteria. Past performance is the one place track record belongs, and it is scored from evidence, not memory.

Calibration: before live scoring, have all evaluators score one sample section (a past proposal or the first criterion of one bid), compare, and settle what a 3 versus a 4 looks like. Ten minutes here saves hours later.

4. Reconcile in a consensus session

• The lead collects independent sheets, then the group reviews criterion by criterion.
• Flag any criterion where evaluator scores for the same bid differ by 2 or more points. Discuss the evidence in the proposal only, then re-score or record a consensus score with the reason.
• Keep independent sheets and the consensus sheet. The gap between them, and why it closed, is part of the record.
• Compute weighted totals with the scorer in bin/score.py, or by hand: weighted total = sum of (criterion score / 5) times criterion weight. The result is a score out of 100.

Consensus is not averaging by default. If the group lands on a number nobody gave, the note must say what evidence justified it.

5. Compare total cost, not headline price

Build one TCO table per finalist on the same horizon and the same usage assumptions. Full checklist in references/rubrics-and-scripts.md. Core buckets:

• Upfront: licence or purchase price, implementation, migration, integrations, hardware, training, internal project time
• Recurring per year: subscription or support, renewals, hosting and storage, admin and support time, upgrades, expected annual uplift, usage-based charges at low, base, and high volume
• End of term: data export and migration out, decommissioning, termination fees, replacement planning

Rules:

• Normalize scope first. If one bid excludes migration and another includes it, add a priced estimate for the missing item or mark the bids not comparable on that line. Never compare a partial price to a full price.
• Do not double count. Training bundled in an onboarding fee is not also a separate training line.
• Show low, base, and high scenarios when usage drives cost.
• Score price from evaluated TCO, not the quote on the cover page. One defensible method, and the one the scorer implements: price points = (lowest evaluated TCO / this vendor TCO) times the price weight. If the RFP promised a different method (points per percent above low bid, average-based, or lowest price gets full points with a floor), use the promised method and document it.
• Run a sensitivity check: drop the price weight by 10 points and raise it by 10. If the winner flips, say so in the recommendation. The decision should survive normal weight changes, or the memo should explain why the close call went the way it did.

Red flags to record, not to auto-reject: a TCO far below every other bid with thin detail, missing recurring lines, an uplift clause with no cap, or exit costs left blank.

6. Check references for finalists only

Run references on the top 2 to 4 bids after scoring, before the award. Use the script in references/rubrics-and-scripts.md. Non-negotiables:

• Same question set, same order, for every vendor
• At least 2 references per vendor from work similar in scope and value, at least 1 live for more than 12 months where possible
• Ask for one reference the vendor did not list if you can source it fairly (a known customer, a public case study contact). Mark which references were vendor-supplied in the notes
• Two callers or one caller plus a recording note, dated, with the reference's role and how long they have used the vendor
• Feed results back into the scorecard: a reference finding either moves a past-performance or risk score, or adds a named risk with a mitigation to the decision memo. A call that changes nothing was not probed hard enough, or it confirmed the score, and the note should say which

Treat these as findings: the same complaint from two independent references, promised versus actual delivery time differing by a wide margin, hidden costs named by a reference that are missing from the TCO table, and a reference who would not renew.

7. Decide, document, and test best value

The highest weighted score is the starting point, not an automatic award. The decision memo (template in references/rubrics-and-scripts.md) must cover:

• Ranking with weighted totals and TCO per finalist
• Strengths and weaknesses that drove the gaps, tied to proposal evidence
• Reference findings and any score changes they caused
• Sensitivity result and any close-call reasoning
• Risks of the recommended vendor and the mitigation or contract term that covers each one
• Why the recommendation is best value, stated in terms of the published criteria. If the recommendation is not the lowest TCO or not the highest technical score, say exactly what the extra cost buys or what the lower score avoids

If demos, presentations, or a pilot sit between scoring and award, score them against criteria published in advance, with the same rubric discipline, and keep those sheets with the record.

8. Debrief and close the file

• Notify all bidders of the outcome at the same time where the rules allow it.
• Offer every unsuccessful bidder a debrief within a stated window. Set the window in advance (federal practice is a written request within 3 days of award notice, but a private buyer sets its own window and should publish it in the RFP).
• A complete debrief covers: that bidder's significant strengths and weaknesses, their evaluated cost and rating next to the awardee's evaluated cost and rating, how their past performance was evaluated, the overall ranking if one was produced, the award rationale in summary, and answers to reasonable questions about whether the stated process was followed.
• Never disclose another bidder's confidential pricing detail, trade secrets, or source-selection deliberations. Evaluated totals that the rules name as debrief content are different from a competitor's line-item bid, keep that line clear.
• After debriefs, log process lessons: criteria that did not predict anything, questions that produced copy-paste answers, weights that made the result hinge on noise. Fix them in the next RFP, not in this record.
• Retain the full file: RFP and amendments, gate results, independent sheets, consensus sheet, TCO tables, reference notes, decision memo, award notice, debrief notes.

Output Contract

Deliver, in this order:

1. Gate sheet: every bidder, pass or fail per mandatory item, clause cited for each fail
2. Scorecard: criteria, weights, per-evaluator and consensus scores with notes, weighted total out of 100 per bidder. The bin/score.py CSV output is an acceptable scorecard
3. TCO comparison: one table, same horizon, low/base/high where usage matters, evaluated TCO per finalist, price points and the formula used
4. Reference summary: per finalist, references contacted, findings, and score or risk changes that followed
5. Decision memo: ranking, evidence, sensitivity result, risks and mitigations, best-value rationale
6. Debrief pack: per unsuccessful bidder, the debrief content listed in step 8, plus a log of debriefs given

Operating Rules

1. Criteria, weights, and the price method are fixed before proposals open. Changes after opening void the fairness of the run; if a genuine error forces a change, cancel, fix the RFP, and reissue.
2. Evidence beats adjectives. Score what the proposal shows (named staff, dated outcomes, priced lines, sample deliverables), not how well it is written. A polished bid with no specifics scores as a vague bid.
3. Independent first, consensus second. Never let the first voice in the room set everyone else's scores.
4. Price is scored on evaluated TCO over the stated horizon. Headline price alone never decides a scored criterion.
5. References are verification, not decoration. Run them before award, on finalists, with a fixed script, and let them move scores.
6. Document as you go. If a score, exclusion, or decision is not in the file with a reason, it did not happen in a defensible way.
7. Conflicts of interest are declared before scoring and recorded. A conflicted evaluator does not score that procurement.
8. Keep bidder information inside the evaluation. No sharing one bidder's content, pricing detail, or standing with another bidder at any point, including debriefs.
9. When scores are close (within about 3 points out of 100), the memo must name the deciding evidence. Close results decided by gut feel are the ones that get challenged.
10. Adapt the trappings to the governing rules. Public-sector buys add formal protest windows, published notices, and records law. Private buys keep the same discipline with lighter paperwork. The scoring, bias, and TCO rules above hold in both.

Helper

bin/score.py computes weighted totals and proportional price points from a CSV of consensus scores, and a TCO total from a CSV of cost lines. See references/rubrics-and-scripts.md for input formats and worked examples.