simulated_buyer_panel v1.0.0

name: "simulated_buyer_panel" description: "Test an offer on a simulated buyer panel before launch: build buyer cards from real customer evidence, run separate scenario interviews with price and tradeoffs forced, tally buy and pass reasons in the buyers' own words, and read the buy rate as an upper bound that a real test must confirm. Use when someone wants to pre-screen a product, price, or positioning idea with synthetic buyers, pilot a concept test, or decide which segment and rival framing a real test should target."

Simulated Buyer Panel

Purpose

Run a cheap, honest pre-screen of an offer before any real launch spend. A panel of simulated buyers, each built from real customer evidence, reacts to the same offer at the same price, under the same script. The panel sorts segments, surfaces objections in buyer language, and kills weak offers early. It does not validate demand, and it never replaces a real test.

The evidence base is blunt about the limits. Argyle and coauthors showed language models can mirror human subpopulations when conditioned on rich backstories from real respondents. Wang, Morgenstern, and Dickerson showed the same models misportray and flatten identity groups, based on human studies with 3,200 participants across 16 identities. A review of about 30 silicon sampling papers from Northeastern's AI-Media Strategies Lab concluded models are unreliable human substitutes and belong in pilot and prioritization work, with results checked against human data. Nielsen Norman Group found synthetic respondents can match a headline conclusion while collapsing the variation real people show. This skill is built to get the pilot value while blocking the known ways a panel fools its owner. Full sources sit in the run notes for this skill's research, and the failure list with controls is in references/bias-controls.md.

Workflow

1. Decide whether a panel is the right tool

Use a panel when all of these hold:

• The decision is early and reversible: which segment to target first, which framing a real test should use, whether an offer is worth a real test at all.
• You have real customer evidence to build cards from: interview notes, support tickets, sales calls, reviews, churn notes.
• A cheap real test exists after the panel, and you are willing to run it.

Do not use a panel when any of these hold:

• The decision is a launch, a price change on a live product, a large build, or anything hard to reverse. Go straight to real buyers.
• You have no real evidence about the buyer. Cards built from imagination produce confident fiction.
• The buyer group is small, local, or poorly represented in public text. The model will hand you a mainstream average wearing their name.
• Nobody will run the real test after. A panel with no follow up test becomes the decision, and that is the failure this skill exists to prevent.

2. Write the decision and the bar before anything else

Write down, before building a single card:

• The decision the panel informs, in one sentence.
• The offer stimulus exactly as buyers will see it, with the real price.
• The bar: the rate and segment pattern that would send you to a real test, and the pattern that would kill or rework the offer. Set the bar from your own economics, for example the conversion a paid test would need to pay back. No universal bar exists in the research, so any borrowed threshold is fiction.
• The real test you will run next if the panel clears the bar.

If the bar is not written first, the panel will always pass, because the reader will move the bar to the result.

3. Build buyer cards from real evidence

Build 12 to 24 cards using the template in references/buyer-card-and-interview.md. Every card carries job context, a recent trigger, the current workaround and its cost, past buying behaviour, budget authority, a hard constraint, a known objection from a real source, and a stance going in.

Rules:

• Demographics alone do not make a card. In controlled tests, demographic only prompting scored near zero agreement with human responses, and adding context about experience and perception raised it to moderate agreement. Behaviour and context are the card. Demographics are a label on it.
• Include cards built to pass: a buyer happy with the free workaround, a buyer with no spend authority, a buyer blocked by a review or a contract, a buyer who quit a rival. Aim for a panel that reflects the real market's spread, skeptics included.
• Cover each segment you plan to quote with at least 4 cards. Under that, the segment gets reasons only, no rate.
• Write the source next to each field. A field with no source is a guess, and a card that is mostly guesses gets cut.

4. Run a control offer and a calibration check

• Run one control offer first: a version you know is weak, overpriced, or absurd for this buyer, through the same script. If the control buys at a rate close to the real offer, the panel is grading politely. Stop and fix the cards or the stimulus. Do not report a failed control run as data.
• If this is the first panel for this team or category, also run one offer with a known real outcome, a past launch or a killed product, and score the panel against reality. Record the result as the panel's calibration record. An uncalibrated panel gets labelled uncalibrated in the report.

5. Run the scenario interviews, one buyer at a time

Use the script in references/buyer-card-and-interview.md, in order, with identical wording and the identical stimulus for every member. Run members separately. Never show a member another member's answer.

The script forces the things simulated buyers skip on their own:

• Past behaviour before opinion: what the problem cost them last time, what they did.
• A tradeoff with a price: this offer, the free current setup, or a named rival at its real price. One choice, with a reason.
• The strongest blocker, the approval path, and who else has to say yes.
• A Juster probability: 0 means no chance, about 1 in 100, 10 means certain, about 99 in 100, with the steps between read as chances in 10. The Juster scale beat plain intention scales at predicting real purchases in the original research and in later replications, because it lets a buyer state a small real chance instead of a polite no.
• A hard verdict: buy or pass today, at the stated price. "Would consider it" is recorded as a pass with reason code no-commit.
• The main reason in the buyer's own words, recorded verbatim before any summary is written.

Strip sales adjectives from the stimulus. A concept test measures the offer, and a pitch for a stimulus gets graded as a pitch.

6. Tally with the tool, reasons first

Record one row per member in a CSV with columns buyer, segment, verdict, juster, reason_code, reason, then run:

python3 scripts/panel_tally.py responses.csv --bar 0.40 (set the bar to the value pre-registered in step 2)

The tool reports the raw buy rate with a Wilson 95% interval, the Juster expected rate (the mean stated probability), segment rates, reason tallies, and flags for unanimity, thin segments, small panels, segment flattening, and repeated verbatim reasons. Worked data and output are in examples/worked-panel.csv and references/worked-example.md.

Read the output in this order:

1. Control and flags. Any blocking flag ends the read. Fix the panel, do not interpret around a flag.
2. Pass reasons. A repeated structural pass reason (no authority, security review, free workaround is good enough, wrong workflow) is a finding even when the rate looks fine. Structural reasons outrank rates.
3. Segment pattern. A segment that buys for one clear reason, next to segments that pass for structural reasons, is a targeting finding: narrow the offer, do not average it away.
4. The rate, last and least. Carry forward the lower of the raw buy rate and the Juster expected rate as the upper bound.

7. Read the buy rate as an upper bound, and say why

Three separate effects push a panel rate above real buying:

• Stated intent overstates behaviour even for real humans. Intent scale translations discount stated scores, in one multi study translation a stated near certain score mapped to roughly four fifths expected actual buying. That discount is an illustration of direction, not a constant to apply.
• Simulated buyers add agreement and positivity bias on top of that, praising concepts real buyers reject.
• Simulated buyers feel no budget pain, no switching cost, and no career risk, so friction is understated unless the card and script force it in.

So report the rate as "upper bound, simulated" in every document that quotes it, next to its interval. On a 16 member panel, 7 buys is 44% with an interval from about 23% to 67%. A report that prints 44% without the interval and the label is misreporting the run.

8. Hand off to a real test, or kill

Close the run with one of three verdicts, matched against the pre-registered bar:

• Proceed to a real test: name the test, the segment, the price, and the behavioural anchor. Good anchors ask for something with a cost: a deposit, a paid pilot, a waitlist signup from a page that names the price and the free alternative. Opinion questions are not a real test.
• Rework: the pass reasons name a fixable offer problem, such as packaging for a segment, a missing integration, a price structure that breaks team budgets. Rework that one thing and rerun the panel once before any real spend.
• Kill: pass reasons converge on a structural blocker across segments, or no segment clears the bar. Write the kill reason down so the idea does not return next quarter wearing a new name.

A panel result by itself never authorizes building the full product, setting a final price, or announcing anything.

Output Contract

A panel run delivers a single report with:

1. The label "Simulated panel, not customer research" at the top, the decision the panel informed, and the pre-registered bar.
2. The offer stimulus exactly as shown, including price, plus the control offer and its result.
3. The panel roster: card count per segment, the evidence source behind the cards, and the calibration record or the word uncalibrated.
4. The tally: panel size, buys and passes, raw rate with its interval, Juster expected rate, the upper bound carried forward, and segment rates where a segment has 4 or more cards.
5. Buy and pass reason tallies with the top verbatim reasons quoted exactly as recorded.
6. Bias flags from the tool, and what was done about each one.
7. The verdict: proceed to a named real test, rework one named thing, or kill with the reason. The real test, when named, includes its behavioural anchor and the result that would confirm or refute the panel.

Operating Rules

• Label every panel output as simulated, everywhere it gets quoted. Passing panel numbers off as customer research is the one unforgivable move in this method.
• More members do not fix a biased panel. Every member shares the same model and its biases. Cap the panel at 24 and spend the effort on card evidence and the real test.
• Never quote a segment rate from fewer than 4 cards, and never quote a rate at all from fewer than 12 members. Small panels produce reasons, not rates.
• Fact check factual claims made by members about rivals, prices, and rules before they enter a report. A member's opinion is panel data. A member's fact is an unchecked assertion.
• One variable per rerun. Change the price, the segment, or the framing, not all three, or the second panel teaches nothing about the first.
• Keep the verbatim reasons. Summaries drift toward what the reader wanted to hear. The buyers' own words, even simulated buyers working from real evidence, are the least processed signal the run produces.
• Quality over cadence: if the evidence to build honest cards does not exist, stop and say so. Collecting three real buyer interviews first is a better use of a day than running a panel on invented buyers.