Agent scratchpad

On this page 7

An agent scratchpad is the running record of intermediate reasoning, tool calls and tool results that an AI agent keeps within its context while it works through a multi-step task. In the field of AI agents, the scratchpad is what allows a language model, which has no internal state between calls, to see what it has already tried and decide the next step.

The term became common with ReAct-style agents, where each iteration appends a thought, an action and an observation to the prompt. Agent frameworks often expose it as a named variable, and it is the short-term working memory of a single run, as opposed to long-term agent memory that persists across sessions.

How an agent scratchpad is built during a run

An agent scratchpad grows by appending one step record per iteration of the agent loop. The harness sends the model the system prompt, the user task and the current scratchpad, the model replies with reasoning and a tool call, the harness executes the tool, and the result is appended as an observation before the next call.

Because large language models are stateless between requests, the scratchpad is the only reason a model on step twelve knows what happened on steps one to eleven. In chat-based APIs it usually takes the form of alternating assistant and tool messages, while older text-completion agents serialized it as a block of labeled lines.

The table below lists the typical entries that make up a scratchpad and the role each one plays.

Entry typeProduced byPurpose
Thought or planModelStates intent and reasoning for the next action
Action or tool callModelNames the tool and its structured arguments
ObservationHarnessReturns the tool output or an error message
Intermediate noteModelRecords findings, open questions or a todo list
Final answerModelEnds the loop and leaves the scratchpad
Typical scratchpad entries

Some systems also let the agent write a persistent notes file to disk, such as a progress log in a coding agent. That file acts as an external scratchpad that survives context resets, which blurs the boundary with agent memory.

The agent scratchpad is often confused with agent memory, agent state and the agent trajectory. All four describe information about a run, but they differ in lifetime, audience and purpose.

ConceptLifetimeMain consumerTypical content
Agent scratchpadOne task or runThe model itselfThoughts, actions, observations
Agent memoryAcross sessionsFuture runsFacts, preferences, summaries
Agent stateOne run, often checkpointedThe orchestration codeVariables, step counters, flags
Agent trajectoryStored after the runEvaluators and developersComplete logged sequence of steps
Chain of thoughtOne model responseThe model itselfReasoning before a single answer
Scratchpad versus neighboring concepts

The scratchpad is closest to the trajectory, but the trajectory is a log for observability and evaluation, while the scratchpad is the live input the model reads. A trajectory is usually complete and immutable, whereas a scratchpad may be trimmed, summarized or rewritten while the run continues.

Managing scratchpad size and context limits

The main engineering problem with an agent scratchpad is that it grows with every step and eventually competes with the context window. Tool outputs such as web pages, file contents or database rows are the usual cause, because a single observation can contain thousands of tokens.

Long scratchpads cause three problems: higher cost per call, higher latency, and degraded attention to early instructions, sometimes described as context rot. The table below summarizes the common techniques for keeping the scratchpad useful.

TechniqueHow it worksTrade-off
Observation truncationCuts tool outputs to a fixed lengthMay drop the decisive detail
Context compactionSummarizes older steps into a short noteLoses exact wording and values
Tool result clearingRemoves stale observations already usedModel cannot re-read them later
External notes fileAgent writes key findings to storageRequires discipline in the prompt
Subagent delegationChild agent explores, returns a summaryExtra calls and coordination cost
Scratchpad management techniques

In practice, production agents combine several techniques. A coding agent may clear old file reads, keep a todo list in its notes and compact the conversation once usage passes a threshold such as 70 to 80 percent of the context window.

Design choices for a reliable scratchpad

A well-designed scratchpad makes the agent’s situation legible to the model at every step. The format of observations, the presence of explicit plans and the handling of errors all influence whether the next decision is sound.

  • Return structured observations with clear status fields, so the model can tell success from failure.
  • Keep error messages informative, because the model uses them to correct its next tool call.
  • Encourage an explicit plan or checklist that the agent updates as items are completed.
  • Preserve the original task at the top of the context, not only inside early scratchpad entries.
  • Mark summarized sections clearly so the model does not treat a summary as a raw tool output.

With reasoning models, part of the scratchpad is hidden reasoning that the provider may return in encrypted or summarized form. Passing these reasoning blocks back on later turns, when the API requires it, keeps the model’s chain of decisions consistent across tool calls.

Risks and common failure modes

Because the agent trusts what is in its scratchpad, errors and untrusted content inside it propagate to later steps. The scratchpad is therefore both a reliability concern and a security boundary.

SymptomLikely causeMitigation
Agent repeats the same tool callObservation unclear or truncatedReturn explicit results and errors
Agent forgets the original goalTask buried under long outputsRestate goal, compact older steps
Agent follows instructions from a web pagePrompt injection in an observationTreat tool output as data, limit permissions
Agent reports facts it never verifiedEarlier hallucinated note reusedRequire tool evidence for claims
Costs rise sharply on long tasksUnbounded scratchpad growthTruncation and context compaction
Scratchpad failure modes

Prompt injection is the most serious risk: text retrieved from an email, document or website lands in the scratchpad with the same visibility as genuine instructions. Robust agents combine model-level resistance with least-privilege agent permissions and human approval for irreversible actions.

Explained without jargon

Think of a person solving a long puzzle with a notepad beside them. Each time they try something, they write down what they did and what happened. Before the next move, they glance at the notepad so they do not repeat failed attempts.

An AI agent works the same way, except it forgets everything between moves. The scratchpad is its notepad, and the agent can only remember what is written there. If the notepad becomes too long or messy, the agent starts to lose track, so good systems keep it short and tidy.

A typical use case

A developer asks a coding agent to fix a failing test in a repository of about 300 files. The agent searches for the test, reads two source files, runs the test suite and edits a function. Each search result, file excerpt and test output is appended to its scratchpad, which grows to roughly 60,000 tokens after 25 steps.

To stay within budget, the harness clears file contents the agent has already acted on and keeps a short progress note listing the cause found and the files changed. When the second test run passes, the agent reads that note and writes an accurate summary. The result is a correct fix in one session without the agent rereading the whole repository.