One Big Prompt vs Many Small Prompts: Which Is Cheaper for AI Coding?

A Question With No Universal Answer

When you have a batch of related coding work — say, adding docstrings to 20 functions — you can ask the AI in one big prompt ("document all of these") or twenty small ones ("document this function"). Developers assume one is obviously cheaper. In reality, it depends on how much context the tasks share, and getting it wrong can double your bill either direction.

The Case for One Big Prompt

Batching wins when the tasks share a lot of common context. Suppose all 20 functions live in one file and the model needs to see the whole file to document any of them. With 20 separate calls, you resend that entire file 20 times as input — paying for the same context over and over.

Bundle them into one request and you send the file once. If the file is 8,000 tokens, that is the difference between 8,000 input tokens (one call) and 160,000 (twenty calls). On a $5/M input model, roughly $0.04 versus $0.80 — a 20x difference driven entirely by not repeating shared context.

The Case for Many Small Prompts

Splitting wins in the opposite situation — when tasks are independent and don't share context, and especially when failure is likely. Three reasons small prompts can be cheaper:

• Cheaper retries — if one item fails or comes back wrong in a batch of 20, you may have to re-run the whole expensive batch. With small prompts, you only re-run the one that failed
• Right-sized output — a giant batch produces a giant response; if the model gets cut off by an output limit mid-batch, you pay for a truncated mess and must redo it
• Model routing — small independent tasks can be routed to a cheaper model, while a monolithic prompt forces everything onto one (often premium) model

The Decision Framework

Ask two questions:

• How much context do the tasks share? High shared context → batch, to avoid resending it. Low shared context → split.
• How likely is failure or partial success? High failure risk → split, so retries are cheap and localized. Reliable, deterministic work → batch.

The worst outcome is a huge batch that shares little context and fails partway — you pay for enormous output, get an unusable partial result, and redo the whole thing.

The Caching Wildcard

Prompt caching changes the calculus. If your provider caches a stable context prefix, the "resend the file 20 times" penalty of small prompts largely disappears — the file is billed at full price once and at a steep discount on the 19 repeats. When caching is available and well-configured, many small prompts become far more competitive with batching, while keeping the retry and routing advantages. Check whether your workflow is hitting the cache before assuming batching is required.

A Practical Default

For most AI coding work: batch tasks that share heavy context and are reliable; split tasks that are independent or failure-prone. Use the async batch API for large, non-urgent batches to capture additional discounts, and lean on caching to soften the cost of splitting when context is shared.

The Takeaway

There is no blanket answer — the cheaper choice hinges on shared context, failure risk, and whether caching is in play. Batch to avoid resending shared context; split to keep retries cheap and enable model routing. Estimate both approaches for your specific workload with our AI coding cost calculator before committing.