How prompt caching works
Before a model writes a single output token it has to process the whole input, computing intermediate key-value states for every token. OpenAI's documentation describes prompt caching as preserving that state for a reusable prefix — the unchanged tokens at the beginning of a prompt — so that a later request starting with the same tokens can reuse it instead of computing it again. The cache stores those internal states, not your text, and the model still writes a fresh answer every time.
The match is exact and runs from the start. Anthropic orders a request as tools, then system prompt, then messages, and states that a change at any level invalidates that level and everything after it: edit one tool definition and nothing downstream is reused. OpenAI likewise requires the entire rendered prefix to match. That is why stable content goes first — instructions, tool schemas, reference documents — and the part that changes on every call, usually the user's question, goes last.
A prefix also has to be long enough to qualify. Anthropic's minimum is 512 tokens on its newest models and up to 4,096 on some older ones; OpenAI's is 1,024 tokens from GPT-5.6 onward; Google lists 2,048 to 4,096 depending on the Gemini model. Shorter prompts are simply processed at the full rate.
How cached tokens are priced
A cached request splits its input into three meters. At Anthropic the usage object reports cache_read_input_tokens (served from cache), cache_creation_input_tokens (written to cache on this call) and input_tokens (after the last cache breakpoint, not cached), and total input is the sum of the three.
- Reads are the discount. Anthropic bills a cache hit at 0.1× the base input price on most models, 0.05× on Claude Opus 5.5 and Sonnet 5.5, and 0.025× on Claude Fable 5.1. OpenAI bills reads at 0.1× uncached input from GPT-5.6 on, and 0.05× on GPT-6.1 Sol.
- Writes can cost extra. Anthropic charges 1.25× base input for a five-minute cache and 2× for a one-hour cache; OpenAI charges 1.25× from GPT-5.6, while its earlier models write for free. The cache write entry covers when that premium pays back.
- Storage is a separate meter on some APIs. Google bills explicit context caches per million tokens per hour of storage, on top of the discounted read rate.
DeepSeek prices input as a cache hit or a cache miss, with the hit rate far below the miss rate. Google turns implicit caching on by default for Gemini 2.5 and newer models and passes the saving through automatically when a request hits the cache.
In our indexCache-read rates in our index
Counted at build from our price index: the published cached-input rate against the standard input rate among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-11 and 2026-10-09. How we collect prices.
How long a cache lives
Caches expire, and an expired prefix is a full-price miss. Anthropic's default cache lives five minutes and is refreshed at no cost each time it is read; a one-hour lifetime is available at the higher write price. OpenAI keeps a GPT-5.6-or-later cache for at least 30 minutes after the latest write or reuse, while earlier models stay in memory typically five to ten minutes when idle, or up to 24 hours with extended retention on the models that support it.
Where a cache lives also matters. OpenAI notes that cached states sit on individual machines, that caches are never shared across organizations, and that traffic above about 15 requests per minute for one prefix can overflow to machines without the entry. A cache-friendly prompt can still miss if requests land somewhere else.
When caching pays — and when it does not
Caching pays when the same long prefix is sent many times within its lifetime: a chatbot with a fixed system prompt, an assistant answering questions about one document, an agent that resends its whole history on every step. The longer the shared prefix and the more often it is reused, the larger the share of input billed at the read rate.
It does little when prompts are short, when every request is different from the first token, or when traffic is so sparse that the cache expires between calls. And where a vendor charges for writes, a prefix that is written and never read costs more than not caching at all. Anthropic notes that on its five-minute cache, one read is enough to pay back the write.
There is a throughput benefit too. Anthropic states that cache hits are not deducted from your rate limit, and for most Claude models only uncached input counts toward the input-tokens-per-minute limit, so a high hit rate lets the same tier process far more total input.
Worked exampleA document assistant with and without the cache
50,000 requests a month on Claude Sonnet 5.5, priced at its current list rates, checked on 2026-10-09.
Each request sends a 20,000-token document plus a short question. In the cached columns the uncached share is priced at the five-minute cache-write rate, the same convention our calculator and model pages use: a miss is the request that writes the prefix again. Output is identical in all three — caching never touches it.
Change the workload or the model in the LLM API cost calculator.
Prompt caching is not response caching
Prompt caching reuses computation on the input. The model still runs, still generates a new answer, and still bills every output token. A response cache — sometimes called semantic caching — is something you build yourself: it stores whole answers and returns a saved one when a similar question arrives, skipping the model call entirely. The two stack: a response cache handles exact repeats, prompt caching makes every call that does reach the model cheaper.