Reflexion

Also known as: verbal reinforcement

On this page 7

Reflexion is a technique for AI agents in which a language model evaluates its own failed attempts, writes a short verbal self-critique and stores it in memory so that the next attempt at the same task is conditioned on that lesson. The method is also described as verbal reinforcement, because feedback improves behavior through text in the prompt rather than through updates to model weights.

Reflexion was introduced in 2023 research on language agents and demonstrated on coding, multi-step reasoning and text-based decision tasks. Its central idea is simple: turn a sparse success or failure signal into a specific, reusable explanation of what went wrong, and give the agent another chance with that explanation in context.

How Reflexion works inside the agent loop

Reflexion wraps a standard agent loop with three roles that may be served by the same model. The actor produces a trajectory of thoughts and actions, often in the style of ReAct prompting. The evaluator scores the outcome, and the self-reflection model converts the score and the trajectory into a natural language lesson.

The lessons accumulate in an episodic memory buffer that is prepended to the actor prompt on the next trial. The loop repeats until the evaluator reports success or a trial budget, commonly three to five attempts, is exhausted. No gradient step occurs at any point, so the model itself is unchanged.

The table below describes the components and what each contributes.

ComponentInputOutput
ActorTask, tools and stored reflectionsTrajectory of thoughts and actions
EvaluatorTrajectory and final resultBinary or scalar feedback signal
Self-reflection modelFailed trajectory and feedbackShort verbal lesson for next trial
Episodic memoryAccumulated lessonsContext added to the actor prompt
Components of a Reflexion agent

Feedback sources that drive reflection

The quality of the evaluator decides whether Reflexion helps. With a reliable external signal, such as unit tests for code, the reflection can name the exact failing case. With a weak signal, such as the model judging its own answer, reflections may confidently reinforce mistakes.

Evaluator typeExampleReliability
Execution feedbackUnit tests, compiler errorsHigh when tests are correct
Environment rewardTask completed in a simulated houseHigh but often sparse
Exact-match checkAnswer compared with referenceHigh where references exist
Heuristic rulesRepeated action or loop detectedModerate
LLM self-evaluationModel grades its own answerLow to moderate
Evaluator types and their reliability

In coding experiments, a useful refinement is to let the agent generate its own tests before reflecting. This improves feedback when no tests are supplied, but self-written tests can be wrong, so a failed generated test does not always indicate a faulty solution.

Reflexion is distinct from single-pass self-refinement, which revises one answer inside a single attempt, and from reinforcement learning, which changes weights over many episodes. Its defining feature is learning across trials of the same task through persistent text memory.

TechniqueWhat changesPersistenceTypical cost
ReflexionPrompt via stored lessonsAcross trials of a taskSeveral full attempts
ReAct promptingReasoning interleaved with actionsWithin one attemptOne trajectory
Self-refineOne output revised iterativelyWithin one attemptA few revision calls
Plan-and-execute agentExplicit plan before actingWithin one attemptPlanning plus execution
Reinforcement learningModel weightsPermanentLarge training compute
Reflexion and neighboring techniques

These methods combine well. A common configuration uses ReAct as the actor, Reflexion as the outer retry loop and a plan-and-execute structure for long tasks, with agent memory holding reflections between trials.

When to use Reflexion

Reflexion is most useful when a task can be retried safely and checked automatically. It suits code generation, puzzle solving, structured data extraction and sandboxed tool use. It is a poor fit where actions are irreversible, because a failed attempt in the real world cannot simply be discarded.

The decision table below summarizes typical situations.

SituationRecommendationReason
Code with a test suiteUse ReflexionTests give precise, trustworthy feedback
Sandboxed agent taskUse with a trial capRetries are cheap and isolated
Open-ended writingPrefer self-refine or reviewNo objective success signal
Payments or production changesAvoid retries by trialFailed attempts have real effects
Latency-critical chatAvoid or limit to one retryEach trial adds a full attempt
Decision guide for applying Reflexion
  • Keep reflections short and specific, naming the failing step and the corrective action.
  • Cap the memory buffer so old lessons do not crowd out the task description.
  • Log every trial in agent observability tooling to audit what the agent learned.

Limitations and failure modes

The main weakness is dependence on the base model’s ability to diagnose errors. Smaller models often write generic reflections such as “be more careful”, which add tokens without changing behavior. Even strong models sometimes attribute failure to the wrong step and then repeat the real mistake.

Costs also grow quickly: each trial multiplies token usage and latency, and the memory buffer lengthens the prompt. Reported gains on benchmarks are real but vary with model, task and evaluator, so they should be measured on the target workload. Finally, lessons are usually task-specific and not automatically transferred; reusing them across tasks requires deliberate memory design and can spread incorrect generalizations.

In everyday language

Reflexion works like a student who fails a practice test and writes a short note about what went wrong. Before trying again, the student reads the note. The next attempt avoids the same mistake.

An AI agent using Reflexion does the same thing in words. It tries a task, checks the result and writes itself a lesson. Nothing inside the model is retrained; it simply carries its notes forward. The method works best when there is a clear way to tell success from failure, such as a test that passes or fails.

Seen in the real world

A software team uses a coding agent to fix small bugs reported in its issue tracker. For each issue, the agent writes a patch inside an isolated sandbox and runs the existing unit tests. On the first attempt, one patch fixes the reported bug but breaks a date-parsing test because it assumes a fixed time zone.

The agent reflects on the failing test output and stores the lesson that dates must be parsed with explicit time zone handling. On the second trial it rewrites the function accordingly and all 214 tests pass. The team limits the loop to three trials per issue and sends any unresolved issue to a human reviewer with the reflections attached, which shortens review time because the reviewer can see what the agent already tried.