How a batch API works
Instead of one HTTP request per prompt, you hand the provider a whole job. OpenAI's Batch API takes a .jsonl file uploaded through the Files API, one request per line with a custom_id, and the only completion window it accepts is 24 hours. Anthropic's Message Batches API takes the requests directly in the API call. Google's batch mode accepts inline requests up to 20 MB in total or a JSONL file of up to 2 GB, with a target turnaround of 24 hours.
Results come back in whatever order the provider finished them, which is what the custom_id is for. Requests the provider could not reach before the window closed expire: Anthropic does not bill expired requests, and OpenAI charges only for the requests that completed.
How the batch discount works
OpenAI describes a 50% discount against its synchronous APIs, Anthropic a 50% discount on both input and output tokens, and Google prices batch usage at 50% of the standard interactive cost for the equivalent model. Not every vendor halves: xAI lists a 20% batch discount on four Grok models, applied to input, output, cached and reasoning tokens, and no discount on the rest.
Discounts stack. Anthropic states that its prompt-caching multipliers combine with the batch discount, and Google supports context caching inside batch requests. A long shared prefix — the same instructions on every item — therefore earns both the batch rate and the cached-read rate. Anthropic recommends its one-hour cache for this, since a batch can take longer than five minutes to work through.
The panel below counts the models in our index whose vendor publishes a per-model batch rate. A vendor can run a batch endpoint without listing per-model batch prices, so the count is what is published, not what exists.
In our indexBatch rates in our index
Counted at build from our price index: published batch rates against the same model's real-time rates among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-21 and 2026-10-09. How we collect prices.
Limits and turnaround
- OpenAI: up to 50,000 requests and 200 MB per batch, up to 2,000 batch creations per hour, and a per-model cap on queued input tokens. Batch rate limits are a separate pool, so a large job does not eat into your real-time limits.
- Anthropic: up to 100,000 requests or 256 MB per batch, whichever comes first. Most batches finish within an hour; results are available once everything completes or after 24 hours, and stay downloadable for 29 days.
- Google: 24-hour target turnaround, with many jobs finishing faster.
The 24 hours is a ceiling, not a schedule. If something downstream depends on the results by a fixed time, build in the full window.
Worked exampleClassifying a backlog in real time vs as a batch
200,000 requests a month on GPT-6 Sol, priced at its current list rates, checked on 2026-09-30.
Two hundred thousand support tickets, each a 1,200-token prompt and a short label back. The work and the token counts are identical; only the rate changes.
Change the workload or the model in the LLM API cost calculator.
When to use batch — and when not to
Batch fits work where nobody is waiting on each answer: tagging or classifying a backlog, extracting fields from documents, generating embeddings for a corpus, running an evaluation set, summarizing yesterday's tickets overnight. It does not fit chat, coding agents that act on each response before sending the next request, or any feature with a user on the other end.
Between real-time and batch sit two related options. OpenAI's flex processing bills at batch rates on the regular endpoints in exchange for slower responses and occasional unavailability. DeepSeek instead discounts by the clock — see off-peak pricing.