1M Token Context Windows: When They Save Money and When They Don't

Million-token context windows sound like a cheat code: dump your entire codebase into the prompt and let the model figure it out. Models like Kimi K3 ($3/$15 per million tokens, 1M context) and Gemini 2.5 Pro ($1.25/$10 per million tokens, 1M context) make this technically possible. But "possible" and "cost-effective" are very different things.

This post breaks down the real math: when stuffing a massive context window saves you money by eliminating retrieval overhead, and when it's just burning cash on irrelevant tokens.

The Cost of a Full Context Window

Let's start with the uncomfortable math. A single API call that fills 500K tokens of context:

That $1.50 per call for Kimi K3 at 500K context adds up fast in an agentic coding loop. A 50-turn session with half a million tokens per turn is $75 in input costs alone. The question isn't whether you can fill the window — it's whether you should.

When Large Context Windows Save Money

There are real scenarios where paying for a huge context is cheaper than the alternative:

1. Eliminating multi-turn retrieval loops. Without a large context window, coding agents play a costly game: search for file → read file → realize it needs another file → search again → read again. Each turn compounds input tokens because the entire conversation history gets re-sent. If your codebase has 200K tokens of genuinely interconnected code, stuffing it in one shot can be cheaper than 15 retrieval turns where the agent re-reads growing context each time.

2. Complex cross-file refactoring. When you need a model to understand type relationships across 30+ files simultaneously — renaming interfaces, updating imports, fixing cascading type errors — a single large-context call often beats iterative discovery that burns tokens on repeated context.

3. Cache-hit scenarios with repeated structure. Kimi K3's cache-hit pricing drops input to $0.30/M tokens. If you're making repeated calls against the same codebase context, subsequent calls cost 90% less. A 500K context call goes from $1.50 to $0.15 on cache hit — that changes the entire calculus.

When Large Context Windows Waste Money

1. Padding with irrelevant files. This is the biggest trap. Developers dump their entire repo into context "just in case." If only 50K out of 500K tokens are actually relevant, you've wasted $1.35 per call on Kimi K3 — paying for tokens the model mostly ignores.

2. Small, isolated tasks. Fixing a typo in one file doesn't need your auth module, database schema, and test utilities in context. A 10K-token call at $0.03 becomes a 500K-token call at $1.50 for zero quality improvement.

3. Repeated large prompts without caching. If you're making one-off calls that can't benefit from cache hits, every call pays full price. Ten unique 500K-token calls on Kimi K3 = $15 in input alone. The same work with targeted 50K-token prompts = $1.50 total.

The Math: Precision vs. Padding

Here's a concrete scenario. You're building a feature that touches 5 files (50K tokens of relevant code) in a 500K-token codebase:

The targeted approach is 10x cheaper per call. But if targeted retrieval takes 8 turns of searching (each re-sending growing context), the full-dump single call might win. This is the core tradeoff.

Practical Rules for Context Budgeting

Based on the pricing math, here are rules that actually work:

• Rule 1: Use the "relevance ratio" test. If less than 20% of your context is directly relevant to the task, you're overpaying. Trim aggressively.
• Rule 2: Cache or don't fill. Large context windows only make economic sense for repeated calls (cache hits) or single-shot tasks where retrieval would cost more in total turns.
• Rule 3: Compare total session cost, not per-call cost. A $1.50 single call that avoids a 10-turn retrieval loop ($0.15 × 10 turns with growing context ≈ $1.00+) might be worth it. Do the math for your specific workflow.
• Rule 4: Stage your context. Start with targeted context for simple tasks. Escalate to full context only when the model explicitly fails to find what it needs.
• Rule 5: Favor models with cache-hit pricing for agent loops. Kimi K3's $0.30/M cache rate makes repeated large-context calls 90% cheaper. If your workflow involves iterating over the same codebase, this pricing structure is dramatically more cost-effective than flat per-token rates.

The Bottom Line

Million-token context windows aren't inherently wasteful or cost-saving — it depends entirely on your relevance ratio and call patterns. The models that pair large windows with aggressive cache pricing (like Kimi K3 at $0.30/M on cache hits) fundamentally change the economics compared to flat-rate models.

The winning strategy: be intentional. Know exactly how much of your context is working for you, and use caching to amortize the rest.

Want to estimate how context window size affects your project's cost? Try our free AI coding cost calculator — input your project parameters and compare models with different context strategies side by side.