The Batch API discount isn't 50% -- and it isn't always a discount

Every guide to cutting LLM costs has the same third bullet: use the Batch API, it’s 50% off. It is the easiest advice in the genre — one flag, half the bill, no quality trade-off, no prompt engineering.

It is also the advice most likely to be quietly wrong for your workload, for two reasons that appear in no vendor’s marketing. The discount is not 50% everywhere. And half of a full-price token is frequently more than you are paying right now.

Every claim below was checked against the vendor’s own pricing page or API documentation on August 21, 2026, and every dollar figure is computed from our pricing snapshot. The short version: batch is the right lever for a narrower set of jobs than the internet thinks, it is the only lever for a set of models nobody mentions, and the thing that decides the answer is usually not the batch discount at all.

One model, four lanes

Start with the shape of the thing, because most people have never looked at it. OpenAI’s pricing page puts a toggle above its flagship table: Standard · Batch · Flex · Fast mode. Same model, same weights, same quality — four different prices per million tokens.

Here is GPT-5.6 Sol, short context, read straight off that toggle:

Then a second axis crosses it. Above 272K input tokens the same table switches to long-context columns at roughly double: standard long context on Sol is $10.00 input, $1.00 cached, $45.00 output.

Multiply the two axes and a single million input tokens on one model has a published price anywhere from $0.25 to $25.00 — batch, short context, cached at the bottom; fast mode, long context, cache write at the top. A hundredfold spread, and not one token of it involves changing model.

The equivalence worth memorising from that table: a long-context batch request costs exactly what a short-context synchronous one does. Both are $5.00 per million input on Sol. If you are pushing 300K-token prompts and flinching at the long-context rate, batch puts you back at the price you were mentally budgeting.

Flex is the batch discount without the wait

Look again at the first two rows: Batch and Flex are the same four numbers. Flex is a synchronous call — you set service_tier to flex and get a response — priced at batch rates.

The trade is capacity, not latency. Flex requests can come back with a 429 resource_unavailable, and the documentation is explicit that “You will not be charged when this occurs.” So the choice is not “half price or fast” but which form of unreliability you can absorb: a 24-hour wait, or a request that sometimes has to be retried or dropped back to standard.

For anything with a deadline measured in minutes rather than days — an hourly enrichment job, a queue that drains overnight but not on a fixed schedule — Flex is the lane people skip because they were looking for the word “batch.”

What each vendor actually gives you

Every number here was read off the vendor’s own page or docs on August 21, 2026.

Two rows there break the rule everyone repeats.

xAI charges 20%, and mostly zero. Its pricing page says the size of the discount “varies by model,” lists exactly four — Grok 4.3 and three Grok 4.20 snapshots — at 20%, and then closes the door: “Models not listed above have no batch discount.” That includes Grok 4.6, the current flagship. If your cost model assumed the industry 50% and your traffic sits on 4.6, you have budgeted a discount that does not exist.

The same page runs the dial the other way too. Priority processing is billed “at a 2x premium over standard rates,” it applies to every token type, and — a detail worth copying into your own billing code — “You are only billed at the priority rate when the response confirms service_tier: "priority". If the request is served at the default tier instead, standard rates apply.” Priority is also not available on batch requests at all.

DeepSeek sells the same discount as a clock. There is no batch endpoint; instead the pricing page states that “Off-peak rates are half of the peak rates” and that “Peak hours are 01:00 - 04:00 and 06:00 - 10:00 UTC (all other hours are off-peak).” That is seven peak hours against seventeen off-peak — 71% of the day is already half price, and getting the discount is a scheduling decision with no code change at all. On DeepSeek V4 Pro that is $1.32/$3.96 peak against $0.66/$1.98 off-peak. Our snapshot stores the peak rates deliberately, so every comparison on this site understates DeepSeek for seventeen hours a day rather than overstating it for seven.

The number the 50% is measured against

Here is the part that turns the advice around.

Take a workload batch is genuinely built for: classify or enrich 100,000 documents a month. A 20,000-token shared prefix — system prompt, taxonomy, few-shot examples — plus 1,000 tokens of unique document, plus a 200-token structured answer.

Five ways to run it, priced off our snapshot:

Read the third row against the second. Batch, applied naively, costs $5,500 against $4,070 — 35% more than doing nothing except keeping your cache warm. The 50% is entirely real: $11,000 to $5,500, exactly half. It is half of the cold price, and nobody serving a shared prefix should be paying the cold price.

The crossover, solved per model: you need roughly a 47% cache hit rate inside the batch before batch beats a warm synchronous path. Below that, the asynchronous version of your job is the expensive one.

Now change one thing about the workload. Remove the shared prefix — independent documents, 2,000 tokens in, 500 out, nothing to cache:

Exactly half, no caveats, no hit-rate risk, nothing to tune. That is the workload the discount was designed for, and it is the one people skip because it does not feel clever.

And on a model with no cache rate published at all, the comparison stops being a comparison. Qwen3.7 Max has no cached-input price; on the first workload it runs $5,400 synchronous against $2,700 in batch. There is no warm path to lose to.

They stack — but “best effort” is a range, not a rate

The two discounts multiply. This is stated, not inferred.

Anthropic: “The pricing discounts from prompt caching and Message Batches can stack, providing even greater cost savings when both features are used together.” OpenAI does not say it in prose but says it in the price sheet — the Batch table carries a cached-input column at $0.25/M on Sol and a cache-write column at $3.125/M, the same 0.1x and 1.25x ratios as the standard lane, on half the base. Google supports it explicitly: “Context caching is supported for batch requests. Reuse cached content by specifying the cached_content resource name.”

What none of that guarantees is that you get hits. Anthropic’s sentence continues: “because batch requests are processed asynchronously and concurrently, cache hits are provided on a best-effort basis.” Between the two ends of its stated 30–98% range, our Opus 5 job moves from $5,025 to $1,115 — a 4.5x spread on identical inputs, decided by scheduling you do not control.

Three things measurably improve the odds, all from the vendor’s own guidance:

1. Put an identical cache_control block in every request in the batch. Not similar — identical, same position, same content.
2. Use the 1-hour TTL. Anthropic recommends it specifically for batches, since “batches can take longer than 5 minutes to process.” A 1-hour write costs 2x base instead of 1.25x, which inside a batch works out at 1.0x base — the 1-hour write in a batch costs exactly the standard synchronous input rate. It pays for itself on the second read.
3. Keep a steady stream of requests so entries do not expire between rows.

And one thing that will not work, because it is blocked: max_tokens: 0 — the cache pre-warming call — is rejected inside a batch. Anthropic gives the reason rather than just the rule: “an ephemeral cache entry written during batch processing would likely expire before the follow-up request runs.” You cannot warm a cache from inside the batch that needs it.

What you give up

The discount is not free of product constraints, and the list is longer than “it’s slower.”

Anthropic rejects six parameters outright, with a validation error rather than a silent downgrade:

Everything else works: vision, tool use, all server tools, multi-turn conversations, extended thinking, and most beta features. Requests inside one batch can be heterogeneous — different models, different shapes — because each is processed independently.

That independence is the actual design constraint, and it is the one people discover after building. A batch is a set of requests that do not know about each other. An agent turn is not independent of the turn before it; you cannot express “call the tool, read the result, decide the next call” as rows in a JSONL file. You can batch a fleet of independent tasks, but each row then runs its own synchronous loop that the Batch API is not doing for you. If a design meeting ends with “and then we’ll just put the agent on batch,” the meeting has a bug.

The other constraints are operational:

• Results expire. Anthropic keeps them 29 days; Google keeps them 6 weeks; then they are gone, even though the batch record itself remains.
• The queue, not the batch, is the ceiling. Anthropic allows 100,000 requests per batch on every tier but caps in-flight requests at 200,000 / 300,000 / 500,000 by tier. OpenAI caps a batch at 50,000 requests and 200 MB and separately meters enqueued prompt tokens per model. xAI limits you to 1,000 add-batch-request calls per 30 seconds.
• Spend limits leak. Anthropic warns that “because of high throughput and concurrent processing, batches may go slightly over your Workspace’s configured spend limit.” If you rely on that cap as a circuit breaker, it is a soft one here.
• Batch requests do not consume your standard rate limits at OpenAI, Anthropic and xAI. This is the underrated half of the feature — moving bulk work to batch buys back synchronous headroom for the traffic that actually has a user waiting, independent of the money.

What actually goes wrong (and what doesn’t)

The fear people bring to batch is expiry: I submit 50,000 rows, the window closes, I lose the job. Price it and it shrinks.

Expired requests are not billed. Anthropic returns an expired result type — “You will not be billed for these requests.” OpenAI writes them to the error file with “This request could not be executed before the completion window expired” and charges only for completed ones. So the cost of expiry is the retry, and if you re-run the expired share f synchronously at full price, your effective rate is 0.5 + 0.5f of standard:

You would need 60% of a batch to expire before a 50% batch discount degrades to the 20% one xAI hands out by default. Partial failure is a real operational problem and a minor financial one. Handle it with a re-submission loop keyed on custom_id and stop budgeting for it.

The thing that actually goes wrong is the cache, and it goes wrong silently. There is no error for a batch that got 30% hits instead of 95%. The job completes, the results are correct, and the invoice is four and a half times what you modelled. The only defence is reading cache_read_input_tokens against cache_creation_input_tokens in the per-row usage and alerting on the ratio — the same discipline that makes caching work synchronously, applied to a surface where you cannot watch it live.

Which workloads, concretely

Batch fits when the requests are independent and the deadline is soft. Embeddings over a corpus. Backfilling a classification onto historical records. Nightly re-scoring. Eval suites. Synthetic data generation. Bulk translation of separate documents. Anything where each row could have been sent a week apart without changing the answer.

Batch fits especially well when you are cold anyway — a monthly job with nothing warm to lose, or a model with no cached-input rate published at all.

Batch does not fit anything with a user waiting, any agent loop, anything needing streaming or fast mode, and — the non-obvious one — any high-volume job whose shared prefix you are already keeping hot. There, batch is a downgrade wearing a discount.

Flex, or an off-peak schedule, fits the middle: work that is too impatient for 24 hours and too cheap to justify standard rates.

A procedure that survives the next reprice

Every number in this post is a vendor policy, and vendor policies moved twice in the two months before it was written. The procedure is what you keep.

1. Find your baseline before you shop for a discount. Compute what the job costs today, at your current cache hit rate. Not the list price — the price you actually pay. Half of the wrong baseline is a wrong answer.
2. Check the vendor’s batch table, not the industry number. Confirm your specific model is on it. xAI is proof that “50%” is a habit, not a rule.
3. Ask whether the requests are independent. If they share a big prefix, you are choosing between two discounts, not adding one. If they do not, batch is free money and you should have taken it already.
4. If they share a prefix and you still want batch, engineer for the hit rate — identical cache_control blocks, the 1-hour TTL, steady submission — and model the 30% case, not the 98% one.
5. Look at the lanes either side of batch. Flex may give you the same rate synchronously. An off-peak window may give it to you with no code change.
6. Instrument the usage object per row, not the invoice per month. Cache ratio and expiry share are the two numbers that decide whether the migration worked.
7. Re-check after every reprice. The crossover point is a ratio of vendor rates. When one of them moves, so does it.

What will break this advice

• The 50% convention is a convention. Nothing forces it. xAI already prices at 20% and zero, and a lab under margin pressure has an obvious dial here that costs it no headlines.
• Batch cache hit rates are best-effort and undisclosed. The 30–98% figure describes current behaviour; it is not a commitment. If scheduling changes, every stacked number in this post moves with it.
• The lanes keep multiplying. OpenAI renamed priority processing to Fast mode on July 30, 2026, and now ships four lanes plus a long-context axis on one table. Expect more axes, not fewer — and expect the copy-the-wrong-column mistake to get easier. We have made it once already.
• DeepSeek’s clock is new. Peak/off-peak billing arrived on August 16, 2026. The hours are a business decision and can be redrawn.

The stable part is the arithmetic. Half of a full-price token is still more than a tenth of one. Whatever the vendors do to the percentages, the first question stays the same: half of what?