intent_routing v1.0.0

name: intent_routing description: Route user intents to the right agent, tool, or skill with a layered classifier, calibrated confidence thresholds, a fallback chain, and a case-based eval harness. Triggers when asked to build or review a routing layer, intent classifier, dispatch system, or agent triage setup.

Intent Routing

Purpose

Turn free-text user input into a routing decision: which agent, tool, or skill handles it, with a confidence score and an explicit action (route, confirm, clarify, fallback). This is the routing block in Anthropic's workflow patterns: classify once, dispatch to a specialist, keep the control flow predictable and cheap.

Workflow

1. Define the target set. Write each target (agent, tool, skill) as one description line: name, what it handles, what it must NOT handle. Keep the set small (3-8 targets). See references/target-catalog.md for the catalog format.
2. Build the layered classifier. Order is fixed: exact/trigger rules first, then a cheap semantic classifier, then a small LLM for ambiguous inputs. See references/router-architecture.md for the layer contract.
3. Set threshold bands. Three actions minimum: route (high confidence), confirm (mid), clarify/fallback (low). Tune the cutoffs on your eval cases, never copy someone else's numbers. See references/router-architecture.md.
4. Wire the fallback chain. Every layer can fail. Classifier timeout, malformed output, empty registry, unknown target: each maps to a safe default, never to a confident wrong route. Full error table in references/router-architecture.md.
5. Evaluate. Build 30-100 labeled cases (request, context, expected target). Run them through scripts/eval_router.py. Read precision per intent, abstain rate, and the confusion pairs. If two targets keep confusing the router, fix the target descriptions, not the router. Guidance in references/eval-kit.md.

Output Contract

A finished routing setup produces:

• targets.yaml: the target catalog (name, handles, not-handles per target).
• A classifier pipeline implementing the layer contract (reference implementation in scripts/router.py).
• thresholds.yaml: the tuned cutoffs with the domain and date they were tuned for.
• evals/cases.jsonl: labeled cases plus the last run report (per-intent precision, overall accuracy, abstain rate).

Operating Rules

1. Fix the taxonomy before tuning the classifier. If the classifier is a coin flip between two targets, the targets overlap: rewrite their descriptions.
2. Route cheap first. The expensive classifier only sees what the cheap layers could not decide. Log which layer decided each route.
3. Never emit a confident wrong route. Below the lowest threshold the router clarifies or falls back. A confident misroute with no fallback class is the classic failure.
4. Carry conversation context into the classifier (last topic, last target). Bare messages like "yes please" are unroutable without it.
5. Thresholds are per-domain and dated. A 0.7 cutoff tuned on support tickets does not transfer to a different target set.
6. Re-run evals after every target addition or threshold change. Log the confusion pairs that moved.
7. Every factual claim about performance (latency, accuracy, cost) comes from a real eval run, not from intuition.