Prompt Caching with Deep Agents: How Teams Cut Agent Token Costs by 41-80%

The 41-80% Claim Is Plausible, But Setup-Dependent

LangChain's prompt caching work for deep agents highlights a cost-reduction range that will get every engineering manager's attention: 41% to 80% lower agent token cost depending on setup. The number is plausible because deep agents repeatedly send large, stable context: system instructions, tool descriptions, memory, project rules, plans, and sometimes retrieved documents. If those repeated tokens are cacheable, the effective price of a long agent loop can fall dramatically.

The catch is that prompt caching is not a magic discount on all tokens. Provider behavior differs. Cache eligibility, cache lifetime, minimum prefix size, pricing for cache reads, and whether tool schemas are included can vary by model and API. Teams that assume a universal 80% discount will overestimate savings. Teams that design their agent prompts around stable prefixes can still capture a large share of the benefit.

Why Deep Agents Are Perfect Caching Candidates

A simple chatbot prompt changes every turn. A deep coding agent has a different shape. It usually starts with a long system prompt, a role contract, tool definitions, repository rules, safety policies, and a task plan. Across a multi-step coding session, much of that context remains unchanged while only the latest observation, tool result, and next instruction changes.

That repeated prefix is where caching works. If the provider can recognize that the beginning of the prompt is identical, it can avoid reprocessing the same tokens at full price. The deeper the agent loop, the more valuable the stable prefix becomes. A one-turn answer may not benefit much. A 30-step coding agent with the same tool inventory and project rules can benefit a lot.

What to Cache in a Coding Agent

A Simple Savings Model

The easiest way to model prompt caching is to split each agent request into two buckets: stable prefix tokens and dynamic tokens. If a request has 80,000 input tokens and 50,000 are the same project instructions, tools, and files as the previous call, those 50,000 are the cache opportunity. If only 10,000 tokens repeat, the opportunity is much smaller.

A rough formula is: effective savings = repeated input share × provider cache discount × cache hit rate. If repeated input is 70%, cache reads are 90% cheaper than full reads, and the cache hit rate is 80%, effective input savings are about 50%. Output tokens are usually not discounted by prompt caching, so total bill savings depend on the input-output mix.

Provider Cache Behavior Must Be Compared Carefully

Different providers expose prompt caching differently. Some require explicit cache control markers. Some automatically cache long prefixes. Some discount cached input heavily but charge more for cache writes. Some have short cache lifetimes, which matters if your agent pauses between steps. Others may not treat tools, files, or system messages exactly the same way.

That means the right benchmark is not just model A versus model B on list price. It is model A with cache hits versus model B without them, or model A's cache-write cost versus model B's lower base input price. A cheap uncached model can beat an expensive cached model for short tasks. For long agent runs, cache behavior can dominate list pricing.

Implementation Rules That Protect Cache Hits

• Put stable content first. System prompt, tool schemas, project rules, and stable memory should precede dynamic observations.
• Do not reorder tools every request. Even harmless ordering changes can prevent prefix reuse.
• Avoid timestamps in the cached prefix. A changing timestamp can make the whole prefix look new.
• Separate memory updates from hot loops. Rewrite memory at checkpoints, not after every tool call.
• Measure cache hits in logs. Do not rely on theoretical savings; track billed cached tokens and full-price tokens.

When Prompt Caching Does Not Save Much

Prompt caching has weak economics when the agent is short-lived, the prompt changes heavily every turn, outputs dominate total spend, or the provider's cache lifetime does not match the workflow. It can also disappoint when retrieval injects different documents every request. In those cases, model routing, smaller context, or better task decomposition may save more than caching.

Teams should run an A/B cost trace before promising 80% savings. Pick five representative agent tasks, record full input tokens, cached input tokens, output tokens, cache misses, and final task success. Then compare total dollars per successful task, not just dollars per request. The AI Cost Estimator can help convert the token mix into a comparable budget.