---
name: "prompt_caching_strategy"
description: "Cut LLM input costs with prompt caching: design cacheable prefix layouts, place cache breakpoints, pick TTLs, measure hit rates, and run break-even cost math across Anthropic, OpenAI, and Gemini. Triggers on LLM cost optimization, caching configuration, or why a cache never hits."
metadata: { "includeInPrompt": true }
---

# Prompt Caching Strategy

## Purpose

Turn a repeated LLM prompt into a cache-friendly layout so identical prefixes get served cheap instead of re-encoded every call. Covers breakpoint placement (Anthropic), automatic caching behavior (OpenAI), explicit context caching (Gemini), hit-rate measurement, and the break-even math that decides whether caching pays off.

## Workflow

1. **Inventory the request shape.** List everything sent on every call, in render order: tool definitions, system prompt, few-shot examples, retrieved documents, conversation history, current user message. Note which parts change call to call.

2. **Pick the provider mechanism.** See [provider specs](references/provider-specs.md).
   - Anthropic: explicit `cache_control` breakpoints (up to 4 per request).
   - OpenAI: automatic, no configuration needed, prefix >= 1,024 tokens.
   - Gemini: explicit `CachedContent` API for long-lived prefixes.

3. **Order the request stable-first.** Put stable content at the top, volatile content at the bottom. The layout that hits:
   ```
   [tool definitions]      <- cache (Anthropic: breakpoint on last tool block)
   [system prompt]         <- cache (Anthropic: breakpoint on last system block)
   [few-shot examples]     <- cache
   [retrieved documents]   <- cache only if reused across calls
   [conversation history]  <- cache up to the last stable turn
   [current user message]  <- never cache; changes every time
   ```
   A single byte difference in the prefix invalidates everything after it. Timestamp, request id, or session token placed early in the prompt silently kills every hit.

4. **Set TTL by traffic rhythm.** 5-minute (Anthropic default, OpenAI automatic) fits chat/agent loops with back-to-back turns. 1-hour (Anthropic `{"type":"ephemeral","ttl":"1h"}`, Gemini `CachedContent`) fits bursty workloads with 20-60 minute gaps. The 1-hour write costs 2x, so it needs 3+ reads to break even.

5. **Prove it works from usage fields, not vibes.**
   - Anthropic: `cache_read_input_tokens` > 0 on the second request, and `cache_creation_input_tokens` = 0 after the first write.
   - OpenAI: `usage.prompt_tokens_details.cached_tokens` > 0.
   - If reads stay at 0, walk [the cache-killer checklist](references/cache-killers.md) before changing anything else.

6. **Run the cost math.** Use `bin/cache_math.py` to compute break-even and savings from real usage numbers. Rule of thumb: cache only prefixes reused at least twice within the TTL window.

## Output Contract

A caching recommendation states: provider, what content sits at each cacheable layer, where breakpoints go (Anthropic), the TTL and why, the break-even point in requests, and how the agent will verify hits from usage fields. If caching does not pay off for a workload, say so and recommend skipping it.

## Operating Rules

1. Verify every factual claim (prices, minimums, TTLs) against current provider docs or pricing pages. Never quote a multiplier from memory in a client-facing recommendation.
2. Below-minimum prefixes silently no-op on Anthropic. Check the minimum for the exact model before assuming a write happened.
3. One write plus one read loses money on Anthropic (1.25 + 0.1 > 1). Never recommend caching a prompt that runs twice or fewer per TTL window.
4. Never recommend parallel fan-out requests expecting shared cache hits: the first response must start streaming before the entry exists.
5. Never let a session token, timestamp, or rotating key sit ahead of a breakpoint. Volatile content goes after the last cache marker, always.