Context compaction is the process of replacing an AI agent’s accumulated context with a shorter, information-preserving version so that a long-running task can continue within the model’s finite context window. In AI agents, the context grows with every user turn, tool call and tool result, and compaction keeps the essential state while discarding or condensing the rest.
The technique is also called context compression. It is a core mechanism of the agent harness, the software around the model that manages prompts, memory and tools, and it determines whether an agent can work for hours without losing track of its goal.
Why AI agents need context compaction
Agents need context compaction because every step of an agent loop adds tokens that the model must reread on the next step. A coding agent that reads files, runs tests and edits code can add tens of thousands of tokens per hour, and even a window of 200,000 tokens or more eventually fills.
A full window is not the only problem. Long contexts raise latency and cost because each request processes all prior tokens, and model accuracy often declines well before the hard limit. Relevant facts buried in the middle of a long history are recalled less reliably, a degradation practitioners sometimes call context rot.
The table below lists the typical sources of context growth in an agent session and how compressible each one is.
| Content type | Typical size | Compressibility |
|---|---|---|
| Tool results | Large, often thousands of tokens | High, mostly reproducible |
| File and web contents | Large | High if they can be refetched |
| Model reasoning and replies | Moderate | Moderate |
| User instructions | Small | Low, must be preserved |
| System prompt and tool schemas | Fixed | Usually not compacted |
How context compaction works
Compaction works by triggering a rewrite of the message history when a threshold is reached and then continuing the session from the rewritten version. Most harnesses trigger it when usage passes a share of the window, commonly 70 to 95 percent, or when the user requests it explicitly.
In the most common method, LLM summarization, the harness sends the history to a model with instructions to produce a structured summary. A good summary prompt asks for the original goal, decisions made, files or resources touched, open problems, and the next planned step. The summary then replaces the older messages, while the system prompt and the most recent turns stay verbatim.
- Monitor token usage after each turn.
- Select the span to compact, usually everything except the latest turns.
- Generate a summary or prune content according to the chosen strategy.
- Rebuild the prompt from the system prompt, summary and recent turns.
- Continue the agent loop and record that compaction occurred.
Compaction is lossy by design. Whatever the summary omits is unavailable to the model afterward unless it can be recovered from an external source, which is why harnesses often pair compaction with agent memory or files on disk.
Strategies for compacting agent context
Context compaction strategies differ in what they remove and how much they cost to run. Simple strategies are cheap and predictable, while summarization preserves more meaning at the price of an extra model call.
| Strategy | How it works | Main trade-off |
|---|---|---|
| Truncation | Drops the oldest messages | Cheap but forgets early goals |
| Tool result clearing | Replaces old tool outputs with stubs | Safe if results can be refetched |
| LLM summarization | Rewrites history as a summary | Preserves meaning, costs a call |
| Hierarchical summary | Summarizes summaries over time | Scales well, detail fades |
| Externalized state | Writes notes to files or memory | Durable, needs disciplined writing |
Tool result clearing is often the first step because tool outputs are bulky and usually reproducible: the agent can read the file again. Production harnesses frequently combine strategies, clearing old tool results continuously and running a full summarization only when the window approaches its limit.
Context compaction compared with related techniques
Context compaction manages the working context of a single session, which distinguishes it from techniques that manage long-term knowledge or split work across agents. The comparison below clarifies the boundaries.
| Technique | Scope | Purpose |
|---|---|---|
| Context compaction | One running session | Fit history into the window |
| Agent memory | Across sessions | Store and recall durable facts |
| Agent scratchpad | Within a task | Hold intermediate reasoning and notes |
| Subagent delegation | Separate contexts | Keep exploratory work out of the main context |
| Agent checkpointing | Saved state snapshots | Resume or roll back execution |
Subagents are a preventive alternative: an orchestrator delegates a search or analysis to a subagent with its own fresh context and receives only a short result. This reduces how often the main context needs compaction, but it does not remove the need for it in long sessions.
Risks and good practice
The central risk of context compaction is losing information that later proves essential. Agents that forget a user constraint, an earlier failed approach or an exact file path after compaction tend to repeat mistakes or violate instructions.
| Symptom | Likely cause | Remedy |
|---|---|---|
| Agent repeats a failed fix | Summary omitted past attempts | Require a list of tried approaches |
| User rule ignored | Instruction summarized loosely | Keep user instructions verbatim |
| Wrong file edited | Paths paraphrased | Preserve identifiers exactly |
| Sudden quality drop | Compaction triggered mid-step | Compact at task boundaries |
Good practice is to compact at natural boundaries, keep identifiers and constraints verbatim, and log every compaction event for agent observability. Teams should also evaluate agents across compaction, testing tasks long enough to trigger it, because short benchmarks never exercise this path.
The short version
An AI agent can only keep a limited amount of text in view at once, much like a desk with room for a fixed number of pages. During a long job, the desk fills with notes, printouts and drafts.
Context compaction is the moment the agent clears the desk but writes a tidy summary first. It keeps the goal, the important decisions and the next step, throws away bulky details it can look up again, and then keeps working with a clean, readable desk.
Where it makes a difference
A developer asks a coding agent to migrate a web application of about 300 source files to a new version of its framework. After two hours, the agent has read dozens of files, run the test suite many times and produced long error logs, and its context reaches about 90 percent of the window.
The harness triggers compaction. Old test logs and file contents are replaced with short stubs, and the remaining history is summarized into a record listing the migration goal, the 38 files already converted, two approaches that failed, the developer’s rule not to change the public API, and the next module to migrate. The context drops to roughly 15 percent of its previous size.
Because the summary preserved the constraints and failed attempts, the agent continues without repeating earlier errors and finishes the migration with the test suite passing.