prompt_patterns v1.0.0

name: "prompt_patterns" description: "Applies proven prompt patterns — few-shot examples, chain-of-thought control (zero/few-shot CoT, self-consistency), machine-parseable output contracts, and tool-use prompt structure — to make LLM outputs reliable. Triggers when a prompt keeps producing flaky, unstructured, or badly-reasoned output; when designing agent system prompts or tool-use flows; or when someone asks for prompt engineering help, before/after prompt rewrites, or how to steer format and reasoning." metadata: { "includeInPrompt": true }

Prompt Patterns

Purpose

Turn unreliable prompts into reliable behavior using a small set of researched patterns instead of vibes-based tweaking:

• Few-shot examples — task format and style demonstrated in-context (the GPT-3 finding: tasks specified via text, no weight updates; Brown et al. 2020).
• Chain-of-thought control — when to elicit step-by-step reasoning, how, and when to skip it (Wei et al. 2022; Kojima et al. 2022).
• Output contracts — prompts that produce machine-parseable results by construction.
• Tool-use prompt structure — XML/delimiter structure, role placement, and long-context ordering that keep agent prompts unambiguous.

Before/after examples live in references/patterns.md; research basis in references/research-brief.md.

Workflow

1. Classify the task. Is it a format/style problem (few-shot), a reasoning problem (chain-of-thought), a parseability problem (output contract), or a confused agent (tool-use structure)? Most real prompts need exactly two of the four — check each in the decision table below.
2. Select patterns. Apply the minimal set: | Symptom | Pattern | |---|---| | Output format drifts, tone inconsistent | Few-shot: 3–5 diverse examples mirroring the real case | | Wrong answers on multi-step problems | Chain-of-thought: "think step by step" (zero-shot) or 2–8 worked exemplars (few-shot CoT) | | CoT still flakes on critical decisions | Self-consistency: sample N paths at temp ~0.7, majority vote | | Parsing the response is fragile | Output contract: JSON/YAML schema + "no prose" rule + parse check | | Agent ignores instructions / mangles tool args | Tool-use structure: XML tags, role in system, data-top/query-last ordering | | Single-fact lookup, translation, sentiment | No CoT at all — it adds tokens and latency for no gain |
3. Compose the prompt. Follow the templates in references/patterns.md:

• Role in one sentence in the system prompt — never as a substitute for concrete instructions.
• Structure with XML tags or markdown headers: <instructions>, <context>, <examples>, <input>; nest hierarchically.
• Long context (20k+ tokens): longform data at the TOP, query/instructions below, the actual question at the END; ask for quotes first before synthesis on multi-document work.
• Output contract: schema first, "return ONLY the JSON" style guard, one example of valid output, forbidden-prose rule.

1. Audit before shipping. Run python3 bin/prompt_audit.py --file <prompt.txt> (optionally --pattern few-shot,contract). It checks: ≥2 examples present for few-shot, schema present for contracts, CoT trigger placed before the answer request, XML/markdown structure consistent, long-context ordering sane. Fix anything it flags.
2. Verify behavior. Generate 3 outputs on real inputs. Pass criteria: all 3 follow the format, reasoning precedes the answer where CoT is used, and at least 2 of 3 are correct on reasoning tasks (self-consistency raises this). Iterate: change one element at a time, keep the examples stable.

Output Contract

A finished prompt package always contains:

• The composed prompt itself (system + user sections clearly marked)
• Which patterns were selected and why (one line each, referencing the symptom)
• The prompt_audit.py result: PASS, or the flags found and how they were resolved
• For reasoning tasks: the 3-output verification note (format adherence, reasoning-before-answer, correctness count)
• A "do not regress" note: the examples and schema are load-bearing — future edits must keep them in sync with the format

Operating Rules

1. Accuracy rule: every factual claim in examples and docs must be verified or framed as opinion. Never invent numbers, dates, quotes, or events. CoT performance claims cite Wei et al. 2022 (8 exemplars; PaLM 540B on GSM8K 17.9% → 56.9%); zero-shot "Let's think step by step" cites Kojima et al. 2022; Anthropic ordering guidance is "their docs' recommendation", not a law of physics.
2. Few-shot examples must be diverse and realistic. 3–5 examples mirroring the real case, covering edge cases. Vary them so the model doesn't latch onto an unintended pattern (wrong thing repeated 5 times is a learned bug).
3. CoT is a tool, not a default. Skip it on single-step lookups (capitals, sentiment, translation): extra tokens, extra latency, no benefit. CoT benefits are emergent at scale — they can hurt small models. Self-consistency is for high-stakes decisions only, where N samples are affordable.
4. Reasoning must precede the answer. A chain of thought written after the answer does not improve it (Wei et al. ablations). Put the CoT trigger before the question is answered, and demand the final answer in a fixed format (Answer: ... or a schema field).
5. Role ≠ instructions. One sentence of role in the system prompt focuses tone; the actual instructions, constraints, and output requirements go with the task. Never hide the real requirements inside a persona.
6. Contracts beat parsing. If downstream code parses the output, the prompt must define the schema, forbid prose outside it, and include one valid example. A post-hoc regex is not a fix for an unconstrained prompt.