AI API Billing Glossary: 20 Pricing Terms Every Developer Should Know

Why the Jargon Matters

AI API pricing pages are dense with terminology, and every term maps to a line on your bill. Misunderstanding one — thinking output is priced like input, or missing that history is resent every turn — is how teams get surprised by a 10x invoice. This glossary defines the 20 terms that most directly determine AI coding costs, grouped by how they affect what you pay.

Core Token Terms

• Token — the unit of billing. Roughly 3-4 characters of English text or a fragment of code. You pay per token, not per word or request.
• Input tokens — everything you send: prompt, system message, attached files, and conversation history. Usually the cheaper rate.
• Output tokens — everything the model generates. Typically 3-6x more expensive than input, so this is where most bills concentrate.
• Tokenizer — the model-specific rule that turns text into tokens. Different models count the same file differently, so per-token prices are not directly comparable.
• max_tokens — the cap you set on output length. A key cost control, but too low triggers truncation and costly retries.

Context and History

• Context window — the maximum tokens (input + output) a model can consider at once, e.g. 128K or 200K. Bigger windows enable bigger prompts but invite bigger bills.
• Conversation history — prior turns resent as input on every new message, because models are stateless. The reason long chats cost quadratically more.
• Context window overflow — when input exceeds the window and older content is dropped or the request fails. Handling it (summarizing, resetting) has cost implications.
• System prompt — instructions prepended to every request. Small per call, but multiplied across thousands of calls it becomes a real line item.

Discount Mechanisms

• Prompt caching — reusing an unchanged prefix (system prompt, files) across calls at a steep discount instead of full input price. The single biggest lever for agentic and multi-turn workloads.
• Cache write / cache read — writing to the cache often costs slightly more than a normal input token; reading from it costs far less. The savings come on the repeat reads.
• Batch API — submitting non-urgent requests for asynchronous processing, commonly at ~50% off, in exchange for slower turnaround.
• Committed-use / volume discounts — reduced rates for pre-committing to a spend level or hitting high usage tiers.

Model and Feature Costs

• Reasoning / thinking tokens — internal tokens some models generate before answering. Billed as output even though you never see them, and they can dominate cost on hard tasks.
• Vision / image tokens — images converted to input tokens, priced by resolution. A single screenshot can cost 1,000+ tokens.
• Tool-call overhead — tool/function schemas and their results count as tokens on every relevant turn, an easily overlooked input cost.
• Per-seat vs usage-based — a flat monthly fee per developer versus metered per-token billing. Which is cheaper depends on how heavily each seat is used.

Limits and Waste

• Rate limit — a cap on requests or tokens per minute. Hitting it triggers retries or backoff, which can waste tokens if not handled cleanly.
• Retry tax — the compounding cost of re-running failed, truncated, or wrong responses. Often the largest source of invisible waste in agent workflows.
• Cost per completed task — total spend (including retries and history) divided by successful outcomes. The only metric that fairly compares models, unlike sticker price per token.

The Takeaway

Every one of these terms is a knob on your bill. The developers who control AI coding costs are fluent in this vocabulary: they know output dwarfs input, that history is resent, that caching and batching are the big discounts, and that only cost-per-completed-task compares models honestly. Bookmark this glossary, and put the concepts to work by modeling your real workload in our AI coding cost calculator.