How to actually get prompt cache hits (and why yours are missing)

Prompt caching is the largest single discount on an LLM bill: a cached token costs a tenth of a fresh one. It is also the only lever on the invoice that fails without telling you. There is no error, no warning field, no degraded-mode log line — just a bill that stays ten times higher than it should be. This is a guide to earning the hit rate, not to the arithmetic of what a hit is worth; that math has its own post. Everything here was verified against vendor documentation on August 13, 2026.

The failure mode is silence

Start with the sentence that explains most zero hit rates. From Anthropic’s caching documentation, on prompts below the minimum cacheable length:

“Shorter prompts cannot be cached, even if marked with cache_control. Any requests to cache fewer than this number of tokens will be processed without caching, and no error is returned.”

Your request succeeds. Your code is correct. You marked the block. And you are paying full price on every call, indefinitely, with nothing in the response to tell you.

That is the shape of nearly every caching bug: the request keeps working. Contrast it with a rate limit, which returns 429, or a context overflow, which returns an error. Caching just quietly bills you more.

So the first rule is diagnostic, not architectural: never assume caching is on. Read cache_read_input_tokens on a real request and confirm it is greater than zero. Everything below is how to make that number stop being zero.

1. The floor, and why it runs backwards

Every vendor has a minimum prefix length below which nothing caches. On Anthropic it varies by model, and the ordering is the opposite of what anyone would guess:

Read the two ends against each other. The flagship, Claude Opus 5, caches from 512 tokens. The budget model, Claude Haiku 4.5, needs eight times more before it will cache anything.

That is exactly backwards from how the models get used. Haiku is the tier people reach for precisely when they have a short prompt repeated at enormous volume — classification, extraction, routing, moderation. It is the workload with the most to gain from caching and the highest bar to clear.

The inversion, priced

Take a concrete service: a 3,000-token system prompt (rubric plus few-shot examples), 200 tokens of user input, a 20-token label out.

The cheapest model on the price sheet is the most expensive model to run this prompt on. Opus 5 lists at 5x Haiku’s input rate and still comes out ahead, because a cached Opus token ($0.50/M) is half the price of a fresh Haiku token ($1/M). Sonnet 5 wins outright at roughly a third of Haiku’s cost.

How far the inversion holds depends on how much the model says back, since output is where Haiku is genuinely cheap. Solving for the crossover on this prompt: Opus 5 stays cheaper than Haiku 4.5 up to 35 output tokens, and Sonnet 5 up to 440. Thirty-five tokens is a label, a score, a small JSON stub — which is exactly the workload Haiku gets picked for. Sonnet’s 440-token margin covers most short-answer work outright. Past those points Haiku wins again, and the whole question stops mattering once you clear its floor.

If you are under the floor, add tokens

The instinct when a prompt is too expensive is to trim it. Below the caching floor that instinct is precisely wrong, because trimming keeps you under the threshold where the 10x discount lives.

Same service on Haiku 4.5, 100,000 requests a month:

Sending 40% more tokens cuts the bill 4.6x. The one-time write costs $0.00525 and is repaid by the eighth request.

To be clear about what this is not: it is not an argument for padding with filler, which costs real money on the write and buys nothing. It is an argument for putting genuinely useful stable content where you were previously economising — more few-shot examples, the full rubric rather than the compressed one, the tool definitions you were tempted to strip. Below the floor those additions are nearly free relative to what crossing the threshold saves.

The alternative fix is to change tier. If the prompt cannot honestly reach 4,096 tokens, the table above says Sonnet 5 is cheaper than Haiku 4.5 for it anyway.

2. Where the breakpoint goes

On Anthropic you have two modes, and picking wrong is the second most common cause of a dead cache.

Automatic caching is one cache_control field at the top level of the request. The system “automatically applies the cache breakpoint to the last cacheable block and moves it forward as conversations grow.” This is what you want for multi-turn chat and agent loops, where the history keeps growing and you want all of it cached.

Explicit breakpoints put cache_control on individual content blocks, up to four. Use these when sections change at genuinely different rates — a system prompt that never moves, a document set that rotates daily, a conversation that changes every turn.

The rule that decides both, quoted from the docs:

“Place cache_control on the last block whose prefix is identical across the requests you want to share a cache.”

Anthropic then documents the mistake people actually make, which is worth reproducing because it is so easy to commit:

“Your prompt has a large static system context (blocks 1 through 5) followed by a per-request block containing a timestamp and the user message (block 6). You set cache_control on block 6… Move cache_control to block 5, the last block that stays the same across requests.”

A breakpoint after a timestamp caches a prefix that will never recur. It is not that the cache misses occasionally — it can never hit, because no two requests share that prefix. If you have exactly one dynamic element in an otherwise static prompt, this single mistake explains a permanent zero.

Order matters, and so does distance

Prefixes are built in a fixed hierarchy: tools, then system, then messages. Each level builds on the previous one, which is why a tool-definition change invalidates everything downstream of it.

There is also a limit almost nobody knows about:

“The lookback window is 20 blocks. The system checks at most 20 positions per breakpoint, counting the breakpoint itself as the first.”

If your messages array has a long tail of small blocks between the breakpoint and the content you meant to reuse, the search can stop before it reaches a valid entry. Structurally, this argues for fewer, larger blocks in the cached region rather than many small ones.

3. What silently kills a live cache

You cleared the floor and placed the breakpoint correctly. Now keep it alive. Anthropic documents exactly what invalidates what, and the blast radius differs:

Three of these deserve calling out because they get changed casually.

Tool definitions are the widest blast radius on the list. They sit at the base of the hierarchy, so editing one tool description invalidates the system prompt cache and the entire conversation cache with it. Renaming a parameter for readability is a billing event. If you ship tool changes on every deploy, every deploy resets every warm cache you have.

Effort and thinking are prompt content, not request metadata. Anthropic is explicit that effort “is rendered into the prompt,” so changing it between turns invalidates the cache, and the documented best practice is to hold it constant within a cached conversation and vary it across workloads instead. Anyone planning to dial effort down mid-conversation to save output tokens is trading a 10x input discount for a smaller output saving — do that arithmetic before shipping it.

Images invalidate on presence, not content. Adding or removing an image anywhere in the prompt breaks the messages cache. A UI that conditionally attaches a screenshot will cache-miss on exactly the turns where the context is longest.

4. The clock is stricter than it looks

Two TTLs: the default 5-minute {"type": "ephemeral"}, and the 1-hour {"type": "ephemeral", "ttl": "1h"}. The prices, relative to base input:

Break-even falls straight out: a 5-minute cache is ahead after one read, a 1-hour cache after two. And reuse is free — “the cache is refreshed for no additional cost each time the cached content is used,” so a steadily-used prefix never pays a second write.

The trap is how the clock is measured:

“The lifetime is measured from the start of the request that writes or reads the cache entry, not from the end of its response. Time spent generating a response counts against the lifetime: if a response takes 4 minutes to stream, a follow-up request that reuses the same cached prefix must start within about 1 minute of that response completing.”

For agents this is the difference between a working cache and a useless one. A long reasoning turn, a slow tool call, a human reading the output — all of it burns the window while nothing appears to be happening. If your loop has any step that can take minutes, the 5-minute TTL is not a five-minute budget; it is five minutes minus however long the model talks. That is the case for the 1-hour cache: not because you need an hour, but because you need the second read to happen at all.

5. Prove it, then alarm on it

Anthropic returns three fields, and the third is the one people misread:

• cache_creation_input_tokens — written to the cache on this request
• cache_read_input_tokens — served from the cache on this request
• input_tokens — only tokens after your last breakpoint, not the total

They sum cleanly, which makes verification a one-liner:

total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

Because input_tokens excludes cached tokens, a naive cost function that multiplies input_tokens by the input rate will understate your bill and hide the fact that caching is not working. The cost function that gets this right needs all three.

Per vendor, the fields to log:

Then set an alert on the ratio. This matters more than it sounds: every invalidation cause in section 3 is a code change you shipped deliberately. Renaming a tool parameter does not look like an incident. It looks like a tidy-up, and the bill triples the next day. A hit-rate metric per deploy turns a silent regression into a visible one.

6. The other three vendors do it for you

Anthropic is the only major vendor that requires you to ask. That makes the other three easier and, in one specific way, harder.

OpenAI caches automatically at 1,024 tokens and up — “for GPT-5.6 and later, 1,024 tokens is a strict minimum” — with hits landing in 128-token increments. Two things changed recently and both cost money: cache writes are now billed at 1.25x the uncached input rate on GPT-5.6 and later (they were free before), and retention moved to prompt_cache_options.ttl, whose only supported value is 30m. Earlier families keep prompt_cache_retention with in_memory or 24h. There is also a routing lever: prompt_cache_key groups requests that share a prefix, with the docs suggesting you keep each key to roughly 15 requests per minute.

Google enables implicit caching by default on Gemini 2.5 and newer — “there is nothing you need to do in order to enable this” — and its minimums are model-specific: 4,096 tokens on Gemini 3.5 Flash and Gemini 3.1 Pro Preview, 2,048 on Gemini 2.5 Flash and Pro. Its two pieces of guidance are the whole technique in one line: “try putting large and common contents at the beginning of your prompt” and “try to send requests with similar prefix in a short amount of time.”

DeepSeek runs disk-based caching “enabled by default for all users, allowing them to benefit without needing to modify their code,” clearing entries “usually within a few hours to a few days.”

The checklist

Run this against any workload you think should be cached.

1. Confirm, do not assume. One real request, read cache_read_input_tokens. Zero means everything below is theoretical.
2. Check the floor for your exact model. 512 on Opus 5, 1,024 on Sonnet 5, 4,096 on Haiku 4.5, 4,096 on Gemini 3.5 Flash, 1,024 on GPT-5.6. Under it, nothing works and nothing complains.
3. If you are under the floor, go up or across. Move stable content into the prefix until it clears, or move to a tier with a lower floor — which, on the numbers above, can be cheaper outright.
4. Put the breakpoint on the last identical block. Not the last block. Anything volatile — timestamps, session ids, the user turn — goes after it.
5. Order the prompt stable-first. Tools, then system, then messages, and within each, everything that never changes before anything that might.
6. Freeze the invalidators. Tool definitions, tool_choice, effort, thinking config, image presence, fast mode. Changing any of them is a billing decision, not a tweak.
7. Choose the TTL from the gap between requests, measuring from request start rather than response end. Under five minutes of real gap, the default wins; over it, 1-hour pays for itself on the second read.
8. Alarm on the hit-rate ratio, per deploy. Every way this breaks is something you shipped on purpose.

None of this is exotic and none of it needs a framework. It is a floor, a breakpoint, a list of things not to touch, and a metric. But it is worth roughly a 10x difference on the largest line of an input-heavy bill, and it is the one lever that will never tell you it stopped working.

Vendor behaviour here was verified on August 13, 2026 against Anthropic’s caching docs, OpenAI’s, Google’s and DeepSeek’s. Rates come from our pricing snapshot and every figure is computed with the engine behind the LLM API calculator. Minimum prefix lengths and cache multipliers are vendor policy and have moved twice in the last two months — if a number here has drifted, tell us.