Context compaction

Also known as: context compression

On this page 7

Context compaction is the process of replacing an AI agent’s accumulated context with a shorter, information-preserving version so that a long-running task can continue within the model’s finite context window. In AI agents, the context grows with every user turn, tool call and tool result, and compaction keeps the essential state while discarding or condensing the rest.

The technique is also called context compression. It is a core mechanism of the agent harness, the software around the model that manages prompts, memory and tools, and it determines whether an agent can work for hours without losing track of its goal.

Why AI agents need context compaction

Agents need context compaction because every step of an agent loop adds tokens that the model must reread on the next step. A coding agent that reads files, runs tests and edits code can add tens of thousands of tokens per hour, and even a window of 200,000 tokens or more eventually fills.

A full window is not the only problem. Long contexts raise latency and cost because each request processes all prior tokens, and model accuracy often declines well before the hard limit. Relevant facts buried in the middle of a long history are recalled less reliably, a degradation practitioners sometimes call context rot.

The table below lists the typical sources of context growth in an agent session and how compressible each one is.

Content typeTypical sizeCompressibility
Tool resultsLarge, often thousands of tokensHigh, mostly reproducible
File and web contentsLargeHigh if they can be refetched
Model reasoning and repliesModerateModerate
User instructionsSmallLow, must be preserved
System prompt and tool schemasFixedUsually not compacted
Sources of context growth in agent sessions

How context compaction works

Compaction works by triggering a rewrite of the message history when a threshold is reached and then continuing the session from the rewritten version. Most harnesses trigger it when usage passes a share of the window, commonly 70 to 95 percent, or when the user requests it explicitly.

In the most common method, LLM summarization, the harness sends the history to a model with instructions to produce a structured summary. A good summary prompt asks for the original goal, decisions made, files or resources touched, open problems, and the next planned step. The summary then replaces the older messages, while the system prompt and the most recent turns stay verbatim.

  1. Monitor token usage after each turn.
  2. Select the span to compact, usually everything except the latest turns.
  3. Generate a summary or prune content according to the chosen strategy.
  4. Rebuild the prompt from the system prompt, summary and recent turns.
  5. Continue the agent loop and record that compaction occurred.

Compaction is lossy by design. Whatever the summary omits is unavailable to the model afterward unless it can be recovered from an external source, which is why harnesses often pair compaction with agent memory or files on disk.

Strategies for compacting agent context

Context compaction strategies differ in what they remove and how much they cost to run. Simple strategies are cheap and predictable, while summarization preserves more meaning at the price of an extra model call.

StrategyHow it worksMain trade-off
TruncationDrops the oldest messagesCheap but forgets early goals
Tool result clearingReplaces old tool outputs with stubsSafe if results can be refetched
LLM summarizationRewrites history as a summaryPreserves meaning, costs a call
Hierarchical summarySummarizes summaries over timeScales well, detail fades
Externalized stateWrites notes to files or memoryDurable, needs disciplined writing
Main context compaction strategies

Tool result clearing is often the first step because tool outputs are bulky and usually reproducible: the agent can read the file again. Production harnesses frequently combine strategies, clearing old tool results continuously and running a full summarization only when the window approaches its limit.

Context compaction manages the working context of a single session, which distinguishes it from techniques that manage long-term knowledge or split work across agents. The comparison below clarifies the boundaries.

TechniqueScopePurpose
Context compactionOne running sessionFit history into the window
Agent memoryAcross sessionsStore and recall durable facts
Agent scratchpadWithin a taskHold intermediate reasoning and notes
Subagent delegationSeparate contextsKeep exploratory work out of the main context
Agent checkpointingSaved state snapshotsResume or roll back execution
Compaction and neighboring agent techniques

Subagents are a preventive alternative: an orchestrator delegates a search or analysis to a subagent with its own fresh context and receives only a short result. This reduces how often the main context needs compaction, but it does not remove the need for it in long sessions.

Risks and good practice

The central risk of context compaction is losing information that later proves essential. Agents that forget a user constraint, an earlier failed approach or an exact file path after compaction tend to repeat mistakes or violate instructions.

SymptomLikely causeRemedy
Agent repeats a failed fixSummary omitted past attemptsRequire a list of tried approaches
User rule ignoredInstruction summarized looselyKeep user instructions verbatim
Wrong file editedPaths paraphrasedPreserve identifiers exactly
Sudden quality dropCompaction triggered mid-stepCompact at task boundaries
Compaction failure symptoms and remedies

Good practice is to compact at natural boundaries, keep identifiers and constraints verbatim, and log every compaction event for agent observability. Teams should also evaluate agents across compaction, testing tasks long enough to trigger it, because short benchmarks never exercise this path.

The short version

An AI agent can only keep a limited amount of text in view at once, much like a desk with room for a fixed number of pages. During a long job, the desk fills with notes, printouts and drafts.

Context compaction is the moment the agent clears the desk but writes a tidy summary first. It keeps the goal, the important decisions and the next step, throws away bulky details it can look up again, and then keeps working with a clean, readable desk.

Where it makes a difference

A developer asks a coding agent to migrate a web application of about 300 source files to a new version of its framework. After two hours, the agent has read dozens of files, run the test suite many times and produced long error logs, and its context reaches about 90 percent of the window.

The harness triggers compaction. Old test logs and file contents are replaced with short stubs, and the remaining history is summarized into a record listing the migration goal, the 38 files already converted, two approaches that failed, the developer’s rule not to change the public API, and the next module to migrate. The context drops to roughly 15 percent of its previous size.

Because the summary preserved the constraints and failed attempts, the agent continues without repeating earlier errors and finishes the migration with the test suite passing.