How Persistent Agent Memory Works: Token Costs of Recall, Decay, and Isolation

Why Agents Need Persistent Memory

A coding agent without persistent memory starts every session from scratch. It re-reads your codebase structure, re-discovers your conventions, and re-learns your preferences — all consuming tokens you have already paid for in previous sessions. Persistent memory solves this by storing and retrieving relevant context across sessions, but it introduces its own token economics that teams need to understand.

Modern agent memory systems (used by tools like Claude Code, Cursor, and custom agent frameworks) typically use a vector store — often Elasticsearch or a dedicated vector database — to persist memories that can be recalled when relevant. The cost is not in storage (pennies per GB) but in the tokens consumed when memories are injected into the context window.

Three Types of Agent Memory

Most persistent memory architectures implement three distinct memory types, each with different token cost profiles:

Episodic memory stores specific past interactions — "last Tuesday, we refactored the auth module and chose JWT over session cookies." These memories are detailed, context-rich, and expensive to recall (typically 200–500 tokens per episode). They are most useful for continuity across multi-session projects.

Semantic memory stores facts and preferences — "this project uses Tailwind v4, the team prefers functional components, tests go in __tests__ directories." These are compact (50–150 tokens each) and high-value because they prevent the agent from asking questions or making wrong assumptions.

Procedural memory stores learned workflows — "to deploy, run tests first, then build, then push to the staging branch." These are medium-length (100–300 tokens) and reduce multi-step task failures by encoding proven sequences.

Token Cost of Hybrid Recall

When an agent starts a session, it performs a recall query against the memory store — pulling in memories relevant to the current task. A typical hybrid recall strategy combines vector similarity search with recency weighting and importance scoring.

The token cost depends on how many memories are recalled and how they are formatted. A conservative recall budget:

• 5 semantic memories × 100 tokens = 500 tokens
• 2 episodic memories × 350 tokens = 700 tokens
• 1 procedural memory × 200 tokens = 200 tokens
• Total per session: ~1,400 input tokens

At Claude Sonnet 4.6 pricing ($3 per million input tokens), that is $0.0042 per session. Seems trivial — but a developer averaging 40 agent sessions per day accumulates 56,000 memory tokens daily. Across a 10-person team, that is 560,000 tokens/day purely for memory recall, or roughly $50/month.

With Claude Opus 4.8 ($5 per million input tokens), the same pattern costs $84/month. With GPT-5.5 ($5 per million input), $84/month as well. The cost scales linearly with team size and session frequency.

Decay Strategies and Their Token Savings

Without decay, memory stores grow indefinitely. An agent that has been running for six months might have 2,000+ stored memories, and the recall system must search through all of them. More critically, stale memories (outdated conventions, deprecated workflows) pollute the context and can cause the agent to generate wrong code.

Time-based decay reduces memory importance scores over time. A memory from three months ago scores lower in recall ranking, making it less likely to be injected. This is the simplest approach — no tokens saved on storage, but fewer stale memories competing for recall slots.

Access-based decay boosts memories that are frequently recalled and deprioritizes those never accessed. This naturally surfaces high-value memories (project conventions accessed every session) while letting one-off context fade.

Consolidation decay periodically merges related episodic memories into compressed semantic summaries. Five separate memories about auth refactoring decisions become one 150-token summary. This can reduce total memory token volume by 40–60% over time while preserving the essential information.

Consolidation requires an LLM call to summarize — typically a cheap model like DeepSeek V4 Flash ($0.1/$0.2 per M tokens) running nightly. A batch of 50 memory consolidations costs approximately $0.02 and can save thousands of tokens in future recall.

Isolation: Preventing Memory Cross-Contamination

In team environments, memory isolation determines whose context gets loaded. Without isolation, one developer's preferences might leak into another's sessions — or worse, memories from a production debugging session might influence unrelated feature work.

Common isolation boundaries:

• Per-user isolation: Each developer's memories are private. Simple, but misses shared project knowledge.
• Per-project isolation: Memories are scoped to a repository or project. Allows team-wide conventions to be shared.
• Hierarchical isolation: Global team memories (coding standards) + project memories (architecture decisions) + personal memories (individual preferences). Most expensive to recall (three queries), but most accurate.

Hierarchical isolation roughly triples the recall token cost because the agent queries three memory stores and injects context from each. The trade-off is fewer errors from missing context — which often saves far more tokens downstream by avoiding incorrect code that needs to be regenerated.

Practical Token Budget for Memory-Enabled Agents

Based on typical team usage patterns, here is what memory costs look like as a percentage of total agent spend:

Memory recall typically represents 3–8% of total agent token spend. The ROI is positive when memory prevents even one unnecessary codebase re-read or convention violation per day — each of which can consume 5,000–20,000 tokens to fix.