What counts as input, what counts as output
Input is the whole request as the model sees it: the system prompt, every earlier turn of the conversation, documents and images, tool definitions and tool results. OpenAI's token guide adds the formatting tokens around messages and roles, which is why a full request counts more than the visible text in it. Every call resends all of this — the model keeps no memory between requests — so input grows with every turn of a conversation.
Output is everything the model generates: the visible answer, any tool calls it makes, and on reasoning models the hidden reasoning tokens. OpenAI states that reasoning tokens are billed as output even though they are not shown as answer text, so a short visible answer can carry a large output bill.
The names differ by API. OpenAI's Chat Completions reports prompt_tokens and completion_tokens; its Responses API and Anthropic report input_tokens and output_tokens.
Why output costs more
A model reads the whole input in one parallel pass, but it writes output one token at a time, each step depending on everything before it. Generating a token therefore occupies the hardware for longer than reading one, and vendors price output accordingly. In our index the output rate is several times the input rate on most current models; the panel below gives the median and the extremes, recounted on every build.
The gap is not fixed across a vendor's line-up or across vendors. Some small models price input and output identically; some flagship models charge many times more for output. That spread is why two models with the same input price can produce very different bills for the same workload.
In our indexOutput vs input rates in our index
Counted at build from our price index: the output rate divided by the input rate, for the 163 models with both above zero, among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-11 and 2026-10-09. How we collect prices.
Which side dominates your bill
It depends on the shape of the work:
- Input-heavy: summarizing documents, answering questions over a knowledge base, classification, and agents that resend a long history on every step. Here prompt caching and trimming context save the most.
- Output-heavy: drafting long text, code generation, and anything on a reasoning model at high effort. Here shorter answers, a lower reasoning effort and a sensible output cap save the most.
Reading your own usage report is the fastest way to tell. If input tokens outnumber output by a hundred to one, a cheaper output rate barely matters; if output is a third of your tokens, it can be most of the bill.
Worked exampleSummarizing vs drafting on the same model
20,000 requests a month on Gemini 3.8 Flash, priced at its current list rates, checked on 2026-09-23.
The summary job moves far more tokens, mostly at the input rate. The drafting job moves fewer tokens, mostly at the output rate. Compare the blended line: the same model is a different price per token for each.
Change the workload or the model in the LLM API cost calculator.