Prompt Caching Explained: How to Cut 90% Off Multi-Turn LLM API Costs

The Multi-Turn Problem: Paying for the Same Tokens Over and Over

Every time you send a message in a multi-turn conversation, the entire conversation history goes back to the model as input tokens. Turn 1 sends 2K tokens. Turn 2 sends 4K. Turn 50 sends 80K. You're paying full price for tokens the model has already seen—and in a coding session, this adds up brutally fast.

Prompt caching solves this by letting the API recognize token prefixes it has already processed. Instead of billing you full price for repeated context, cached tokens are billed at a steep discount—or in some cases, free. For teams running multi-turn coding agents, this is the single highest-ROI optimization available.

How Prompt Caching Works

The mechanism is prefix matching. When you send a prompt, the provider checks whether the beginning of your token sequence matches a recently cached prefix. If it does, those tokens are served from cache instead of being reprocessed.

• Cache write — First request with a new prefix pays full input price and writes to cache.
• Cache hit — Subsequent requests matching that prefix pay a reduced rate (typically 75-90% off).
• Cache miss — If the prefix doesn't match (e.g., you changed the system prompt), full price applies.

The key insight: in multi-turn conversations, every message preserves the previous messages as a prefix. This means cache hit rates naturally increase as conversations get longer—exactly when costs would otherwise spike.

The Math: A 50-Turn Coding Session

Let's model a realistic AI coding session—50 turns with growing context as the agent reads files and generates code. We'll use Claude Sonnet 4.6 at $3/M input tokens with Anthropic's prompt caching (cached reads at $0.30/M—a 90% discount).

Without Caching

With Prompt Caching (90% Hit Rate)

Input cost drops from $12.00 to $2.28—an 81% reduction. Total session cost drops from $15.00 to $5.28 (65% overall savings). The output cost stays the same since caching only affects input tokens.

Provider Comparison: Caching Implementations

Not all prompt caching is created equal. Here's how the major providers differ:

Anthropic's implementation gives the deepest discount (90%) but requires explicit breakpoints. OpenAI's is automatic but only 50% off. OpenRouter's sticky routing ensures your requests hit the same node, maximizing cache hits for whichever underlying provider you choose.

Anthropic Prompt Caching: How to Implement

Anthropic's caching uses cache_control breakpoints in the message array. You mark where the cache boundary should be—typically after your system prompt and any static context. Everything before the breakpoint is cached.

For coding agents, the optimal strategy is: system prompt + file contents as cached prefix, with only the latest user instruction as fresh input. This means your 50K token codebase context is cached across all turns, and you only pay full price for the new 2-4K instruction each turn.

OpenAI Automatic Caching

OpenAI's approach requires no code changes—it automatically detects matching prefixes. The tradeoff: you get 50% off instead of 90%, and you have less control over what gets cached. For GPT-5.6 Sol at $5/M input, cached tokens cost $2.50/M. On our 50-turn example, that turns $20 of input into $11—still significant, but less dramatic than Anthropic's 90% discount.

OpenRouter Sticky Routing

OpenRouter takes a different approach: rather than implementing its own caching layer, it routes repeated requests to the same backend node, maximizing the chance of hitting that provider's native cache. Use the HTTP-Referer header or session tokens to enable sticky routing. This is particularly useful if you're switching between models mid-session—each model's traffic stays on its optimal node.

Maximizing Cache Hit Rates

The difference between 70% and 95% cache hit rate can be thousands of dollars per month at scale. Key techniques:

• Stabilize your prefix — Don't modify system prompts between turns. Every change invalidates the cache.
• Front-load static context — Put file contents, documentation, and schemas before dynamic content like user messages.
• Batch related requests — Keep conversations within the cache TTL window (5 minutes). A 10-minute gap between turns means a full cache miss.
• Avoid unnecessary context shuffling — Some frameworks reorder messages or inject metadata between turns. This breaks prefix matching.

When Caching Doesn't Help

Prompt caching is less effective for:

• Single-turn requests — No repeated prefix to cache. You pay the cache write cost with no subsequent benefit.
• Highly dynamic prefixes — If your system prompt changes every request (e.g., injecting real-time data), cache hit rate will be near zero.
• Output-heavy workloads — Caching only reduces input costs. If your spend is 80% output tokens (e.g., long-form generation), savings are limited.

Real Savings at Scale

Ready to calculate your exact savings with prompt caching? Use our AI cost calculator to model multi-turn sessions with and without caching across Claude Sonnet 4.6, GPT-5.6 Sol, and 90+ other models.