Agent state

On this page 7

Agent state is the complete, structured record of everything an AI agent needs to continue its work at the next step: the goal, the conversation and tool history, intermediate results, plans, counters and the current status of the task. In AI agents, a core area of artificial intelligence, state is what turns a single stateless model call into a multi-step process that can pause, resume, branch and recover.

The underlying language model is stateless between calls: it remembers nothing unless the relevant information is placed in its input again. Agent state therefore lives outside the model, in the agent harness or framework, which decides what is stored, what is shown to the model on each turn and what is persisted for later.

What agent state contains

Agent state contains every piece of information that influences the next decision of the agent. In practice, frameworks organize it as a typed object or dictionary that the agent loop reads at the start of each step and updates at the end.

The components below appear in most production designs, although their names differ between frameworks.

ComponentContentPurpose
Task specificationGoal, constraints, success criteriaKeeps the agent aligned with the request
Message historyUser turns, model outputs, tool calls and resultsProvides context for the next model call
PlanOrdered or hierarchical list of subtasksTracks progress and remaining work
Working dataRetrieved documents, variables, file paths, partial outputsCarries results between steps
Control fieldsStep count, token budget, retries, status flagsEnforces limits and termination
Environment referencesSession IDs, sandbox handles, open browser tabsLinks the agent to external resources
Typical components of agent state

A useful distinction separates explicit state, stored in structured fields the program can inspect, from implicit state, which exists only inside the message history. Implicit state is convenient but hard to validate, query or recover after a failure.

How state flows through the agent loop

State flows through the agent loop as a read, decide, act and update cycle. At each iteration, the harness assembles a model input from the current state, the model proposes an action, the harness executes it and writes the observation and any changes back into state.

Graph-based orchestration frameworks make this explicit: each node is a function that receives the state and returns an update, and reducers define how updates merge, for example appending new messages rather than overwriting them. This design makes state transitions deterministic and testable, even though the model decisions inside them are not.

Because the context window is finite, the harness also performs context selection. Long histories are trimmed, summarized through context compaction, or moved to an external store, so that the state the model sees on each turn is a curated projection of the full state rather than all of it.

Agent state is often confused with agent memory, the scratchpad and the context window, but these terms describe different scopes and lifetimes. The comparison below clarifies the boundaries.

ConceptScopeLifetimeMain role
Agent stateOne task or sessionUntil the task ends or is archivedDrives the next step
Agent memoryAcross tasks and sessionsDays to indefiniteRecalls facts and preferences
Agent scratchpadReasoning within a taskUsually one taskHolds intermediate thoughts
Context windowOne model callA single requestWhat the model actually reads
CheckpointSnapshot of stateUntil deletedEnables resume and replay
Agent state and neighboring concepts

In short, state is the source of truth, the context window is a view of it, memory is long-term knowledge that may be loaded into it, and a checkpoint is a frozen copy of it at a given step.

Persisting and checkpointing state

Persisting agent state means saving it to durable storage after each step, so that a long-running or background agent survives crashes, deployments and waits for human approval. Agent checkpointing stores these snapshots with a thread or run identifier.

Persistence enables several capabilities that are otherwise impossible: human-in-the-loop interruption, where execution pauses until a reviewer approves an action; time travel, where a developer replays or forks a run from an earlier step; and fault tolerance, where a failed tool call is retried without repeating completed work.

The storage choice depends on scale and durability needs, as the table shows.

Storage optionTypical useTrade-off
In-process memoryPrototypes, short interactive runsLost on restart
Key-value storeFast resume of many concurrent sessionsLimited querying
Relational databaseAuditable production workflowsSchema management effort
Object storageLarge artifacts and full transcriptsHigher latency
Event logReplay, auditing, complianceRequires state reconstruction
Options for storing agent state

Design risks and common mistakes

Poorly designed agent state is a leading cause of unreliable agents, because errors in state silently compound over many steps. The table lists frequent symptoms and their remedies.

SymptomLikely causeFix
Agent repeats finished stepsProgress kept only in message textStore an explicit plan with status fields
Quality drops in long runsUnbounded history fills the contextSummarize or offload older history
Resumed run behaves differentlyState depends on in-memory objectsSerialize all state needed to continue
Subagents overwrite each otherShared mutable state without merge rulesUse reducers or isolated substates
Sensitive data leaks into logsCredentials stored in stateKeep secrets in a vault, store references only
Common agent state problems

A further risk is state poisoning: content injected through a tool result, such as a malicious web page, can enter state and influence later steps. Treating tool outputs as untrusted data, validating structured fields and limiting what is persisted reduce that exposure.

The idea in brief

Agent state is the notebook an AI agent keeps while it works. The AI model itself forgets everything between steps, so the surrounding software writes down what the goal is, what has been done, what was found and what comes next.

Imagine a contractor renovating a kitchen over several days. Each morning, the contractor checks the job sheet to see which tasks are finished and which materials have arrived. Without that sheet, the work would start from scratch every day. Agent state is that job sheet, and keeping it accurate is what lets an agent finish long tasks reliably.

A practical scenario

A finance team uses an agent to reconcile about 1,200 monthly vendor invoices against purchase orders. The task takes several hours and includes a pause whenever an invoice differs from its order by more than 2 percent, at which point a human accountant must approve or reject the discrepancy.

The agent state holds the list of invoices with a status field for each, the running totals, the IDs of flagged items and a pointer to the current batch. After every batch of 50 invoices, the harness writes a checkpoint to a database. When the agent reaches a flagged invoice, it records the question in state and stops.

One afternoon, the server restarts while 14 approvals are pending. Because the state was persisted, the agent resumes at invoice 731 the next morning, applies the accountant decisions and finishes the run. No invoice is processed twice and none is skipped, which is exactly what the audit requires.