Agent memory is the set of mechanisms that let an AI agent store, retrieve and update information beyond what fits in a single model call, so that it can carry facts, decisions and experience across the steps of a task, across sessions and across different tasks. In the field of AI agents, memory is what turns a stateless language model into a system that can pursue long goals, personalize its behavior and learn from earlier outcomes without retraining.
A language model on its own remembers nothing between calls: everything it knows about the current situation must be placed in its context window. Agent memory is therefore an engineering layer around the model, built from buffers, summaries, databases and retrieval logic that decide what the model sees at each step.
How agent memory works around a stateless model
Agent memory works by writing selected information to external storage and reading the relevant parts back into the prompt when the agent needs them. Every memory design answers four questions: what to write, where to store it, how to retrieve it and when to forget it.
The write path decides which observations, tool results, user statements or conclusions are worth keeping. Some systems store everything verbatim; others ask the model to extract compact facts or reflections first. The read path assembles the next prompt from the system instructions, recent turns and retrieved memories, usually under a fixed token budget that leaves room for reasoning and output.
Retrieval is typically semantic search over embeddings, often combined with keyword search, recency weighting and an importance score. Some agents also expose memory as explicit tools, such as a call that saves a note or searches past notes, so the model itself decides when to remember and when to look something up.
Types of agent memory
Agent memory is usually divided into short-term and long-term memory, with long-term memory further split by the kind of knowledge it holds. The terms are borrowed loosely from cognitive psychology and describe roles rather than strict technical boundaries.
| Memory type | What it holds | Typical implementation | Lifetime |
|---|---|---|---|
| Working memory | Current task state, recent turns, scratchpad | Context window, message buffer | One task or session |
| Episodic memory | Records of past interactions and outcomes | Logs with timestamps, vector index | Weeks to indefinite |
| Semantic memory | Stable facts about users, domains, systems | Key-value store, knowledge graph, profile | Until corrected |
| Procedural memory | Learned ways of doing tasks | Stored prompts, rules, reusable skills or code | Until revised |
Working memory is the most constrained and the most important: whatever is not in the context window at the moment of a call does not influence the answer. Episodic memory lets an agent recall that a similar request failed last week and why. Semantic memory keeps durable facts such as a preferred language or a project’s deployment target, and procedural memory stores refined instructions or skills that improve future runs.
Key design parameters
The quality of agent memory depends less on the storage engine than on a handful of design parameters that control what reaches the model. The following table lists the most influential ones and their usual ranges.
| Parameter | Typical range | What it affects |
|---|---|---|
| Memory token budget | 10-40% of the context window | Room left for instructions and reasoning |
| Retrieved items per step | 3-20 chunks or facts | Recall versus distraction |
| Chunk or note size | 50-500 tokens | Precision of retrieval |
| Similarity threshold | Tuned per embedding model | Noise filtered out versus facts missed |
| Summarization trigger | When the buffer nears its limit | Loss of detail versus cost |
| Retention policy | Session, days, or until deleted | Privacy, storage cost, staleness |
A common pattern is rolling summarization: older turns are compressed into a running summary while the most recent turns stay verbatim. Another is hierarchical memory, where a small core block always stays in context and larger archives are paged in on demand, much like an operating system moves data between RAM and disk.
Agent memory compared with related concepts
Agent memory overlaps with several neighboring ideas in AI agents and language models, and the differences matter when choosing an architecture. The key distinction is that agent memory is written by the agent itself during operation, while most related mechanisms are prepared in advance.
| Concept | Who writes the content | When it changes | Main purpose |
|---|---|---|---|
| Agent memory | The agent and its runtime | Continuously during use | Continuity and personalization |
| Context window | Assembled per call | Every call | Immediate input to the model |
| Retrieval-augmented generation | Document owners | When the corpus is updated | Grounding in reference knowledge |
| Fine-tuning | Training pipeline | At training time | Changing model weights and style |
| KV cache | Inference server | Within one generation | Speed, not knowledge retention |
In multi-agent systems and agent orchestration, memory can also be shared: a common store or blackboard lets one agent read what another discovered. Shared memory needs clear ownership and conflict rules, otherwise agents overwrite each other’s conclusions.
Choosing a memory design in practice
The right memory design follows from how long the agent must remember and how precise the recall must be. Simple assistants need little more than a buffer; long-running coding agents and personal assistants need several layers.
| Situation | Recommended design | Reason |
|---|---|---|
| Short single-session task | Message buffer only | Context window is sufficient |
| Long conversation | Recent turns plus rolling summary | Keeps cost bounded |
| Returning users | Semantic profile plus episodic search | Personalization across sessions |
| Multi-hour coding or research task | Structured notes file and task list | Survives context resets |
| Regulated data | Scoped stores with retention limits | Supports deletion and audit |
For long tasks, an explicit structured state, such as a progress file, a checklist or a decision log, is usually more reliable than free-text similarity search, because the agent can read it in full after a context reset and continue exactly where it stopped.
Limitations, risks and common mistakes
Memory makes agents more capable but also introduces new failure modes that do not exist in stateless systems. Most of them stem from storing the wrong thing or retrieving the wrong thing at the wrong time.
- Memory pollution: incorrect conclusions get stored and are then retrieved as if they were facts, reinforcing the error.
- Stale facts: preferences or system states change, but old memories still outrank new ones without recency handling.
- Retrieval distraction: too many loosely related items crowd the context and degrade reasoning.
- Memory poisoning: content from web pages or documents is saved and later acts as a hidden prompt injection.
- Privacy exposure: personal data persists longer than intended or leaks between users when stores are not isolated.
- Unbounded growth: without consolidation and deletion, storage cost and retrieval noise rise steadily.
Mature designs therefore add provenance metadata, confidence scores, consolidation jobs and user-visible controls to view, correct and delete memories. Treating memory as untrusted input rather than ground truth is the single most effective safeguard.
The idea in brief
A language model is like a very capable consultant with no notebook: each time you speak to it, it only knows what is written on the page in front of it. Agent memory is the notebook. It lets the agent write down what matters, look it up later and cross out what is no longer true.
Some notes are short-lived, like a sticky note for the current job. Others last, like a client file that records preferences and past decisions. The hard part is not storing notes but choosing which ones to bring to the table each time, because a desk covered with irrelevant papers makes the work worse, not better.
A practical scenario
A software team uses a coding agent to migrate a large codebase to a new framework, a job that takes many hours and far exceeds one context window. The agent keeps a structured progress file listing finished modules, open problems and decisions such as the chosen date library, plus a test log it updates after each run.
Each time its context fills up, the agent’s runtime clears older turns and the agent rereads the progress file before continuing. A separate episodic store records that one module failed twice because of a circular import, so the agent checks for that pattern first in similar modules. The result is a migration that continues consistently across dozens of context resets instead of repeating finished work or reopening settled decisions.