What the context window is
A model has no memory between API calls. Each request has to carry everything the model should take into account — instructions, the conversation so far, documents, tool results — and the context window is the hard ceiling on how much that can be. Anthropic describes it as the model's working memory, distinct from the much larger body of data it was trained on.
The window covers both directions. Anthropic's documentation lists what counts: the system prompt, every message including tool results, images and documents, the tool definitions, and the output generated for the turn, thinking included. OpenAI says the same of reasoning tokens: they are not visible, but they occupy space in the window. A request with a 990,000-token prompt on a one-million-token model has very little room left to answer.
How big context windows are
Windows have grown by orders of magnitude, and a million tokens is now common at the top end. Anthropic gives the Claude models released from Opus 4.6 onward — Sonnet 5.5 and Haiku 5.5 included — a one-million-token window with up to 128,000 output tokens per request, while older ones such as Haiku 4.5 have 200,000. The panel below shows how windows are spread across the current models in our index: the median, how many reach a million tokens, and how many still sit at 128K or less.
The published number is a limit, not a promise of equal quality across it. Anthropic warns that as token count grows, accuracy and recall degrade — it calls the effect context rot — which makes choosing what goes into the window as important as how much fits.
In our indexContext windows in our index
Counted at build from our price index: published context windows, for the 152 models whose vendor states one as a number, among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-11 and 2026-10-09. How we collect prices.
What happens at the limit
Two different things can run out. If the input alone is larger than the window, the request is rejected: Anthropic returns a 400 error saying the prompt is too long. If the input fits but the answer runs out of room, behaviour varies — on Claude 4.5 and newer models the request is accepted and generation stops with stop_reason: "model_context_window_exceeded"; on OpenAI's reasoning models the response comes back with status incomplete, sometimes before any visible text, which is why OpenAI recommends reserving at least 25,000 tokens for reasoning and output when you start with them.
Chat products hide this by dropping or summarizing old turns. On the API it is your job: trim history, clear stale tool results, or use a vendor's compaction feature — Anthropic offers server-side compaction in beta for Claude 4.6 and later models.
What a full window costs
A context window is priced like any other input: every token in it is billed, on every request. A long conversation therefore gets more expensive with every turn, because each new call resends all the turns before it. An agent that runs fifty steps with a growing history pays for the early steps' content fifty times over.
Three things change the arithmetic. Prompt caching bills the repeated part of the window at a fraction of the input rate — though Anthropic is explicit that cached prefixes still occupy the window: caching changes what you pay, not whether the tokens count. Long-context pricing raises the rate for the whole request on some models once the prompt passes a threshold. And retrieval (RAG) avoids filling the window at all, by sending only the passages a question needs.
Worked exampleWhat filling a million-token window costs
100 requests a month on Claude Sonnet 5.5, priced at its current list rates, checked on 2026-10-09.
One hundred requests that each send a 900,000-token codebase and get a 2,000-token answer. The model has no long-context surcharge, so the rate is the standard one; caching is what moves the total.
Change the workload or the model in the LLM API cost calculator.
Why agents fill windows fastest
An AI agent works in a loop: call the model, run the tool it asks for, append the result, call the model again. Every step resends the whole history plus the new tool result, so the window fills with file listings, search results and command output the model looked at once and no longer needs. Tool definitions add a fixed load on top: Anthropic documents that declaring its computer-use toolset adds about 4,500 input tokens to every request, before a single message is sent, and each MCP server connected to an agent contributes its own tool schemas.
The vendors now ship tools for exactly this. Anthropic's context editing clears old tool results and thinking blocks from the history, its tool search tool defers tool definitions until they are needed, and server-side compaction summarizes earlier turns so a session can continue past the window. Each one shrinks what is resent on every step — which is where an agent's input bill comes from.
Choosing a window size
Pick the smallest window that holds your largest realistic request with room for the answer, then design to stay well inside it. Large windows are for genuinely large single inputs — a codebase, a long contract, hours of transcript. They are a poor substitute for retrieval when the relevant material is a small part of a large corpus, because you pay for every token you send and recall degrades as the window fills.