What Is Batch API Pricing? How Async Requests Cut AI Coding Costs 50%

Batch API pricing is the most under-used discount in the LLM market. Google, OpenAI, and Anthropic all charge roughly half their normal rate for asynchronous work — yet most teams pay full price out of habit. If your job doesn't need an answer in the next few seconds, you're likely overpaying by 2x.

The Batch API is a simple deal: you submit a batch of requests, the provider processes them whenever it has spare capacity (within a stated window, usually up to 24 hours), and you pay about 50% of the synchronous rate. You give up latency; you get half off.

Why the Discount Exists

Synchronous requests force the provider to keep GPU capacity on standby to answer instantly. Batch requests don't — they can be slotted into idle time between real-time traffic. That flexibility is worth money, so the provider shares the savings. It's the same idea as off-peak pricing, but the lever is latency tolerance rather than time of day.

The Math on a Real Workload

Say you run a nightly documentation pass over your codebase: 30M input tokens and 6M output tokens on a model priced at $2/$10 per million.

• Synchronous: 30M × $2/M + 6M × $10/M = $60 + $60 = $120/night
• Batch (50% off): $60/night

That's $60/night saved for zero quality difference — the model is the same, the output is the same, you just waited. Over a month that's $1,800 on one recurring job.

Which Coding Workloads Fit Batch

• Great fit: bulk code review, test generation across a repo, documentation generation, changelog summaries, migration/refactor scripts, embeddings backfills, dataset labeling, evaluation runs. Anything queued rather than typed.
• Bad fit: interactive pair-programming, live debugging, autocomplete, anything where a developer or user is waiting. Batch can take minutes to hours to return — unacceptable for real-time work.

Stacking Batch With Other Discounts

The real wins come from combining. Two examples:

• Batch + prompt caching. If your batch shares a stable prefix (system prompt, coding standards, repo context), cache it. A cached input token that's also batched can drop to a fraction of the base rate — on some models a batched cache hit is a quarter of the normal input price.
• Batch + off-peak. A provider that offers both may let a batch job scheduled during off-peak hours capture both discounts. Check whether they stack before assuming.

Practical Setup Tips

1. Audit your recurring jobs. Any cron-driven or nightly LLM task is a batch candidate. Move it first.
2. Design for the window. Submit batches with enough slack that the 24-hour processing window never blocks a deadline.
3. Handle partial failures. Batch jobs return results asynchronously; build retry logic for individual failed items rather than re-running the whole batch.
4. Keep interactive traffic separate. Route real-time requests to the synchronous endpoint and only queued work to batch — don't try to force one path for both.

The bottom line: latency is a cost input you control. If nobody's waiting on the answer, batch it and keep half the money. Estimate how much your workload could save in our AI Cost Calculator.