Agent evaluation is the systematic measurement of how well an AI agent completes multi-step tasks in an environment, judged by its final outcomes, the quality of its intermediate actions, its safety behavior and its resource consumption. Within AI agents as a branch of artificial intelligence, agent evaluation, often shortened to agent evals, differs from classic model benchmarking because the system under test acts, observes and adapts over many turns rather than producing a single answer.
A useful evaluation therefore tests the whole agent harness: the language model, its prompts, tool schemas, memory, permissions and orchestration logic. Two agents built on the same model can score very differently once these components change.
What agent evaluation measures
Agent evaluation measures task success, process quality, efficiency and safety across a set of representative tasks. Because agents are non-deterministic, each task is usually run several times, and results are reported as rates with their variability rather than single pass or fail verdicts.
The table below lists the dimensions most teams track and the metrics that typically express them.
| Dimension | Typical metric | What it reveals |
|---|---|---|
| Outcome success | Task success rate, pass@k, pass^k | Whether the goal state is reached |
| Trajectory quality | Correct tool choice, redundant steps | Whether the path is sound |
| Tool-call accuracy | Valid arguments, schema errors | Interface reliability |
| Efficiency | Steps, tokens, latency, cost per task | Operational viability |
| Safety and policy | Rate of disallowed actions | Risk of harmful side effects |
| Robustness | Success under perturbed inputs | Sensitivity to noise and variation |
Two consistency metrics deserve attention. pass@k counts a task as solved if any of k attempts succeeds, which suits settings where a human picks the best result. pass^k requires all k attempts to succeed, which reflects the reliability demanded in unattended automation and drops sharply when an agent is inconsistent.
How agent evaluations are built
An agent evaluation consists of tasks, an environment, a harness and graders. Each task specifies a starting state and a goal; the environment supplies tools, files, web pages or simulated users; the harness runs the agent loop and records the agent trajectory; graders score the result.
- Define the task set from real usage, including edge cases and tasks that should be refused.
- Build a reproducible environment, ideally an agent sandbox that resets to a known state before every run.
- Specify success criteria as checkable end states, such as a passing test suite or a correct database record.
- Run repeated trials with fixed model versions and logged configuration.
- Grade outcomes and trajectories, then review failures manually to find root causes.
Grading methods vary in cost and trustworthiness, and mature evaluations combine several of them, as the following comparison shows.
| Grader type | How it works | Strength | Weakness |
|---|---|---|---|
| Programmatic check | Inspects final state or runs unit tests | Objective and cheap | Only fits verifiable tasks |
| Model-based judge | A language model scores output against a rubric | Handles open-ended work | Bias and calibration drift |
| Human review | Experts rate outcomes or trajectories | Highest validity | Slow and expensive |
| Simulated user | A scripted or model-driven user interacts | Tests dialogue behavior | Simulator may be unrealistic |
Agent evaluation compared with model benchmarks
Agent evaluation extends model benchmarking from single responses to sequences of actions with side effects. A model can excel on question-answering benchmarks and still fail as an agent because errors compound across steps: a small per-step error rate becomes a large end-to-end failure rate over a long trajectory.
| Aspect | Model benchmark | Agent evaluation |
|---|---|---|
| Unit of test | Single prompt and response | Full multi-turn episode |
| Environment | Static dataset | Interactive tools and state |
| Scoring | Answer correctness | End state, path, cost and safety |
| Variance | Low with fixed decoding | High; repeated runs needed |
| Main failure source | Knowledge or reasoning gaps | Planning, tool use and recovery errors |
Agent evaluation also complements agent observability. Evaluation runs before deployment on curated tasks, whereas observability monitors live traces in production; findings from production traces are a primary source of new regression tasks for the evaluation suite.
Types of agent evaluation in practice
Teams use several evaluation types at different stages of development, each answering a distinct question about the agent.
| Evaluation type | Stage | Question answered |
|---|---|---|
| Capability evaluation | Research and model selection | Can the agent solve hard tasks at all |
| Regression evaluation | Every change to prompts or tools | Did anything that worked break |
| Safety and red-team evaluation | Before release | Does it resist misuse and stay in scope |
| Online evaluation | Production | How does it perform with real users |
Public benchmarks for coding agents, web and browser agents, computer-use agents and customer-service dialogues provide useful external reference points. However, they rarely match a specific product, so most organizations maintain a private task suite drawn from their own workflows.
Pitfalls and limitations
Agent evaluation is difficult to make valid and stable, and several recurring problems cause misleading scores.
- Too few trials: a single run per task hides variance, so differences of a few points are often noise.
- Benchmark contamination: tasks published online may appear in training data and inflate results.
- Reward hacking: agents may satisfy a weak grader, for example by deleting a failing test instead of fixing the code.
- Environment drift: live websites and external APIs change, which breaks reproducibility.
- Unchecked model judges: a judge that is not validated against human ratings can reward fluent but wrong work.
- Ignoring cost: a higher success rate achieved with many times more tokens may not be a real improvement.
The most effective safeguard is regular manual reading of trajectories. Aggregate scores show that something changed, but only transcripts show why, and they frequently reveal grader bugs rather than agent bugs.
In simple terms
Agent evaluation is a structured test drive for an AI agent. It does not stop at one question and one answer. Testers give the agent a real job, such as fixing a bug or booking a meeting. Then they watch whether it finishes. They check how it gets there. They also check whether it does anything risky.
The same agent can behave differently each time. So the test is repeated many times. The result tells a team how often the agent can be trusted with that kind of work and what it costs.
A practical example
A software company is preparing a coding agent that resolves internal bug reports by editing code and opening pull requests. The team collects 150 past bugs that already have fixes and tests, then packages each one in a container that resets the repository to the state before the fix.
Each task runs five times. A programmatic grader runs the hidden test suite, a model-based judge reviews whether the change stays in scope, and engineers manually read 30 failed trajectories. The first version solves 58 percent of tasks on at least one attempt but only 31 percent on all five attempts, and several failures come from the agent editing test files.
After the team restricts write permissions on test directories and improves the search tool description, the all-attempts rate rises to 44 percent at similar token cost. The suite then becomes a regression gate that every prompt or tool change must pass before release.