What a system prompt does
The system prompt is the part of a request the developer writes once and the user never sees: who the model is, what it may and may not do, what format to answer in, and any standing reference material — a product catalogue, a style guide, a policy. Anthropic takes it in a separate system parameter; OpenAI's Responses API takes instructions, and its caching documentation describes developer messages as the place for application instructions.
Vendors add system text of their own, too. OpenAI's prompt-caching guide shows a hidden system message of OpenAI-provided instructions at the very start of the rendered prompt. Anthropic documents that when you pass tools, the API automatically adds a tool-use system prompt — 286 tokens on Claude Sonnet 5.5 and Opus 5.5, more on some older models — on top of your own text and tool definitions.
Why it costs you on every call
LLM APIs are stateless: each request must carry everything the model needs. A system prompt is therefore input on every call, multiplied by your request volume. A long prompt that looks cheap in a playground becomes the largest line on the bill at scale — and in a multi-turn chat or an agent loop, it is resent on every turn alongside a growing history.
That is also why it is the ideal candidate for prompt caching. It is long, identical across requests and sent first — the exact shape a prefix cache needs. Keep it stable: a timestamp, a user's name or a per-request variable near the top changes the prefix and turns every request into a cache miss.
Worked exampleA 4,000-token system prompt at scale
100,000 requests a month on Claude Sonnet 5.5, priced at its current list rates, checked on 2026-10-09.
Each request is a 4,000-token system prompt plus a 300-token user message, with a short reply. The cached column bills most of the system prompt at the cache-read rate; the uncached share is priced at the cache-write rate, as in our calculator.
Change the workload or the model in the LLM API cost calculator.
Keeping it lean
Measure it: run the system prompt through a token counter for the model you use, because the same text counts differently on each vendor's tokenizer. Remove instructions the model follows anyway, examples that no longer earn their tokens, and reference material only some requests need — retrieve that per request with RAG instead. Then cache what remains. The worked example shows the difference caching makes on a long, stable prompt.