Compaction: when to summarize an agent's history — and when it backfires

Every long-running agent eventually faces the same wall: the context window fills up and the run dies. Compaction is Anthropic’s answer — let the model summarize its own history and continue from the summary. It works, and it’s the difference between an agent that finishes a 200-step task and one that suffocates at step 90. But it is a far blunter instrument than it looks: it throws away the original, it fires on a token count rather than a sensible moment, and it quietly bills you for a pass that doesn’t appear in your usage numbers. Here’s the mechanism, the hidden cost, and the specific ways it turns against you.

Compaction is a rewrite, not a trim

The previous post in this series covered context editing, which clears stale tool results out of the window. It’s easy to file compaction under the same heading — “another thing that makes the context smaller” — and that framing will lead you to use it wrong.

The distinction is the whole point:

• Context editing is surgical. It deletes specific spent tool results and leaves everything else exactly as it was. The conversation is still verbatim; it just has some holes where old search results used to be.
• Compaction is a rewrite. It reads the conversation, writes a prose summary, and then discards everything that came before. Nothing is verbatim afterward. You are no longer talking to a model that remembers the conversation — you’re talking to one that read a summary of it.

Anthropic’s docs describe the discard step without euphemism:

“On subsequent requests, append the response to your messages. The API automatically drops all content blocks prior to the compaction block, continuing the conversation from the summary.”

That’s a lossy, one-way operation. It’s the right call when the alternative is death by context exhaustion. It’s the wrong call when you reach for it as a routine cost optimization — because the thing you paid to delete might have been the thing you needed.

How it works

You opt in with a beta header and a config block. Compaction then runs itself:

1. Input tokens cross your trigger threshold.
2. The API generates a summary of the conversation so far.
3. It emits a compaction block containing that summary.
4. It continues the response with the compacted context.
5. On the next request, everything before the compaction block is dropped.

The block itself is unremarkable to look at:

{
  "type": "compaction",
  "content": "Summary of the conversation: The user requested help building a web scraper..."
}

The config surface is small — four fields, and two of them are footguns:

A minimal call:

response = client.beta.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=conversation,
    betas=["compact-2026-01-12"],
    context_management={
        "edits": [{
            "type": "compact_20260112",
            "trigger": {"type": "input_tokens", "value": 150000},
        }]
    },
)

Compaction is a newer-model feature and still in beta. Supported today: Fable 5, Mythos 5, Mythos Preview, Opus 4.8, Opus 4.7, Opus 4.6, Sonnet 5, and Sonnet 4.6.

The bill you don’t see

Here’s the detail that belongs on a pricing site and appears almost nowhere else.

Compaction is not free bookkeeping. Summarizing your conversation means reading your entire conversation and writing a summary — that’s a full extra model call, billed at your model’s input and output rates. Anthropic calls it an additional sampling iteration, and surfaces it in a usage.iterations array:

{
  "usage": {
    "input_tokens": 23000,
    "output_tokens": 1000,
    "iterations": [
      { "type": "compaction", "input_tokens": 180000, "output_tokens": 3500 },
      { "type": "message",    "input_tokens": 23000,  "output_tokens": 1000 }
    ]
  }
}

Look at the two levels. The top-level usage reports 23,000 input and 1,000 output — the post-compaction message, looking wonderfully cheap. The iterations array tells the truth: a compaction pass that chewed through 180,000 input tokens and wrote 3,500 output tokens happened too, and you paid for it. The docs say so outright:

“The top-level input_tokens and output_tokens do not include compaction iteration usage. They reflect the sum of all non-compaction iterations. To calculate total tokens consumed and billed for a request, sum across all entries in the usage.iterations array.”

This reframes the economics. Compaction is not “make context smaller, pay less.” It’s pay a large lump sum now to make every subsequent turn cheaper. It’s an investment with a payback period. If the run ends shortly after a compaction fires, you paid 180k tokens to summarize a conversation you then threw away — a pure loss. The longer the agent keeps working after the compaction, the more turns amortize that cost. Short agents should not be compacting at all. It’s the same shape of trade-off as context editing’s cache re-write, just bigger and more visible once you know where to look. (It’s also a textbook example of the sort of cost that never appears on a price-per-token comparison — see hidden LLM API costs for the rest of that family.)

On caching: compaction and prompt caching do coexist, and the docs give a specific piece of advice worth following. Put a cache_control breakpoint at the end of your system prompt, so the system prompt is cached as its own segment. Then when compaction rewrites the conversation, your system-prompt cache survives and only the new summary has to be written as a fresh cache entry. Skip that and a compaction torches more of your cache than it needs to.

When compaction backfires

Four failure modes, roughly in order of how often they bite.

1. It fires at a token count, not at a good moment. This is the structural weakness. input_tokens is the only supported trigger type — compaction has no idea what your agent is doing when it crosses 150,000. If the agent is halfway through a delicate multi-step derivation, or deep in a debugging loop where the last twelve failed attempts are the whole point, the summarizer will flatten all of it into a paragraph and move on. The threshold knows about size. It knows nothing about whether this is a safe place to forget.

2. It erases the negative evidence. Summaries record what happened; they’re much worse at recording what didn’t work and why. The Manus team, writing up their production agent lessons, make the point that they deliberately keep failed actions and their observations in context, because seeing its own mistakes is what stops the model repeating them. A summary that says “attempted to parse the config” preserves none of the signal from four distinct parsing approaches that failed for four distinct reasons. Compact aggressively and you can watch an agent cheerfully re-attempt something it already ruled out — because from its point of view, it never tried.

3. instructions replaces the default prompt. Read this one twice, because it reads like a convenience and behaves like a trapdoor:

“Custom instructions don’t supplement the default prompt. They replace it completely.”

Pass a well-meaning one-liner like “summarize the key decisions” and you have just discarded Anthropic’s entire tuned summarization prompt — everything it knew about preserving open threads, unresolved state, and task structure. Your summaries get worse, gradually and silently, and the failure shows up much later as an agent that loses the plot after every compaction. If you customize, write a full prompt, not a hint.

4. On short runs it’s pure overhead. Covered above, but it bears repeating as a failure mode: a compaction that fires near the end of a run is 180,000 tokens spent to compress a context you were about to discard anyway.

The research frontier: let the model pick the moment

If the core weakness is timing, the obvious fix is to stop triggering on size and start triggering on state — compact when the agent has just finished something, not when it’s mid-thought.

That’s exactly what a recent preprint proposes. “Self-Compacting Language Model Agents” (arXiv 2606.23525, Li, Zhang, Jurayj et al., submitted 2026-06-22, revised 2026-07-10) gives the model a compaction tool plus a rubric for when to use it: fire when a sub-task resolves or the trajectory is converging; hold off mid-derivation or when stuck. The model decides, not a counter.

The reported results, tested across seven models: up to +18.1 percentage points on math, +5–9 points on agentic search, and 30–70% lower cost per question compared with fixed-interval summarization. Notably, it needs no fine-tuning or external supervision — it’s a prompting-and-tooling result.

You don’t have to wait for the API to catch up, either. pause_after_compaction gives you a manual seam: pause when the summary is written, inspect state, and decide what to preserve before the agent charges on. And nothing stops you from setting a high trigger and calling compaction deliberately at points your own orchestration knows are safe — after a sub-task closes, between phases — rather than letting a token counter choose for you.

Compaction vs context editing vs memory

The three tools overlap enough to confuse and differ enough to matter. Picking by symptom:

The pairing that matters most: memory plus compaction. Compaction is lossy, so the mitigation is to make sure the important things were written down somewhere durable before the summarizer got to them. Anthropic’s own docs recommend exactly this combination for long-running agents — compaction keeps the active context small without client-side bookkeeping, and memory preserves what must survive summarization. If you’re going to compact, give the agent a place to save its notes first.

Tuning it

A short, opinionated starting point:

1. Fix your accounting first. Sum usage.iterations. Until you do, every conclusion you draw about whether compaction helps is based on numbers that omit its cost.
2. Set the trigger high, not low. The 150,000 default is reasonable; the 50,000 floor is not a target. Every compaction is a large lump-sum cost, so you want few of them, late, on runs long enough to amortize.
3. Cache your system prompt separately. A cache_control breakpoint at the end of the system prompt keeps it out of the blast radius.
4. Add memory before you add compaction. Give the agent somewhere to record state that a summary would flatten.
5. Leave instructions alone unless you’re prepared to write a complete summarization prompt — remember it replaces, not extends.
6. Reconsider whether you need it. If your runs finish comfortably inside the window, compaction is a solution to a problem you don’t have.

The pattern across this whole series is the same: none of these levers is free, and each trades something you can measure (tokens) for something you can’t easily measure (fidelity, timing, detail). Compaction makes that trade at the largest scale — it’s the one that buys the most room and takes the most away. Use it when you need the room.

Verified on 2026-07-11 against Anthropic’s official documentation: compaction docs, context editing docs, memory tool docs, and effective context engineering. Beta headers, parameter defaults, the model list, and the usage.iterations behavior are quoted from those pages; the self-compaction results are from an un-peer-reviewed preprint and are labelled as such. Production practice on retaining failures is reported by the Manus team. API betas change often — the live docs win. This is the second post in a series on context engineering; the first covered context editing. See our methodology for how we verify.