model_switching_guide v1.0.0

name: "model_switching_guide" description: "Decide whether switching to a stronger LLM model mid-run is worth it: cost/quality tradeoff math, pilot capability checks before switching, and clean handoffs to the new model. Use when a run is underperforming and a bigger model is being considered." metadata: { "includeInPrompt": true }

Model Switching Guide

Purpose

Make the switch-models decision on evidence, not hope, and hand the task to the new model without losing anything.

Workflow

1. Diagnose before switching. A stronger model only helps when the failure is a capability gap: the output is coherent but keeps missing the same class of nuance, reworded prompts fail the same way, the task needs reasoning the current model keeps fumbling. If the failure is a bad prompt, missing context, or a broken tool, switching models wastes money and fixes nothing.

1. Measure the current model first. Take 3 to 5 tiny subtasks that represent the real work. Run them on the current model and score pass/fail against one rubric you will reuse for the new model. That is your baseline. Never use benchmark scores from memory; measure on your own tasks.

1. Do the arithmetic. Compare expected total cost per finished task, not cost per request. All figures below are illustrative examples, not real provider prices.

• attempt cost = rate x tokens
• expected cost with retries = attempt cost / success rate
• expected total = attempt cost + (1 - success rate) x downstream cost of a wrong answer
• break-even: switch only when the real cost of a wrong answer exceeds (attempt_B - attempt_A) / (success_B - success_A)

Worked example (computed, illustrative): 25k tokens per attempt, Model A at 1 unit/M tokens with 50% success, Model B at 10 units/M tokens with 90% success, downstream cost of a wrong answer at 5 units. Attempt cost: A=0.0250, B=0.2500. With retries: A=0.0500, B=0.2778. With downstream cost: A=2.5250, B=0.7500, so B wins. But on an easy task (98% vs 99% success, downstream cost 0.2): A=0.0290, B=0.2520, so stay on A. Break-even downstream cost for the hard case: 0.5625 units. Below that number, the cheap model wins even with its misses.

Save-then-run worksheet. Save as switch_decision.py, edit the variables, run python3 switch_decision.py:

python
   # Illustrative figures only. Not real provider prices.
   tokens_k = 25     # tokens per attempt, in thousands
   rate_A = 1.0      # illustrative cost units per 1M tokens, model A (cheaper)
   rate_B = 10.0     # illustrative cost units per 1M tokens, model B (stronger)
   s_A = 0.50        # success rate of A on pilot subtasks (measure it yourself)
   s_B = 0.90        # success rate of B on the same pilots
   D = 5.0           # illustrative cost of a wrong answer slipping downstream

   attempt_A = rate_A * tokens_k / 1000
   attempt_B = rate_B * tokens_k / 1000
   total_A = attempt_A + (1 - s_A) * D
   total_B = attempt_B + (1 - s_B) * D
   breakeven_D = (attempt_B - attempt_A) / (s_B - s_A)

   print(f"attempt cost:   A={attempt_A:.4f}  B={attempt_B:.4f}")
   print(f"expected total: A={total_A:.4f}  B={total_B:.4f}")
   print(f"verdict:        {'switch to B' if total_B < total_A else 'stay on A'}")
   print(f"break-even D:   {breakeven_D:.4f} (below this, stay on A)")
   

Real output from this exact script:

   attempt cost:   A=0.0250  B=0.2500
   expected total: A=2.5250  B=0.7500
   verdict:        switch to B
   break-even D:   0.5625 (below this, stay on A)
   

1. Pilot the new model before committing. Run the same 3 to 5 subtasks through the new model, score with the same rubric. Switch only the task class that failed, not the whole run: keep everything the cheap model handles on the cheap model.

1. Hand the task to the new model cleanly. The handoff packet needs: the goal in one paragraph, hard constraints, what has already been tried and why it failed (so the new model does not repeat the dead ends), what is decided and finished, what is still open, the grading rubric, and only the context the task needs, never the whole session dump. Frame it as a fresh assignment. Send facts, not commentary about the previous model.

1. Verify after switching. Spot-check the first outputs against the rubric. If the new model does not beat the old model on the same pilots, switch back. You measured, so act on it.

Output Contract

A switch decision record with four items: the diagnosis (capability gap, one line), the measured baseline and pilot numbers, the arithmetic with its verdict, and the handoff packet if switching. No switch happens without all four.

Operating Rules

1. Never quote real provider prices or benchmark scores. Cost figures are illustrative; capability numbers come from your own pilot subtasks.
2. Stay at the run-strategy layer. Per-request retry logic, API failover, and provider outage handling are out of scope.
3. Move only the failing task class to the stronger model. Leave the rest of the run where it is.
4. Keep the handoff packet small and current: goal, constraints, failures to avoid, open items, rubric. Never dump the whole session.
5. Not this skill: for passing context between agents or sessions, use context-handoffs: "How to hand context between agents or sessions. Use when a session is getting long, when work resumes after a pause, when one agent hands work to another, or when someone asks what the next agent needs to keep going." For testing prompt or model versions with scored cases, use eval-harness: "Test AI prompts and model behavior like code: build a small case suite with scorers, run prompt variants against it, and require the new version to beat or tie the old one on pinned regression cases before it ships." This skill owns only the run-strategy call to switch models mid-run.