Error analysis

On this page 7

Error analysis is the systematic inspection of the cases a machine learning model gets wrong, with the goal of grouping failures by cause, estimating how much each cause contributes to the total error and deciding which fix to pursue next. In model evaluation, it complements aggregate metrics: a score such as accuracy or F1 says how often a model fails, while error analysis explains why and where it fails.

The practice applies to classifiers, regression models and large language models alike. Its output is not a single number but a ranked list of failure categories, each with a frequency, an example set and a hypothesis about the underlying cause.

How error analysis is carried out

Error analysis follows a repeatable workflow that turns a pile of mistakes into a prioritized plan. It starts with a representative sample of errors, typically 100 to 300 cases drawn from a validation set, not from the test set reserved for final reporting.

  1. Collect errors: run the model on a labeled evaluation dataset and gather misclassified or low-scoring outputs.
  2. Read examples: inspect each case with its input, prediction, reference and any metadata.
  3. Tag causes: assign one or more failure tags, starting with open coding and refining into a stable taxonomy.
  4. Count and size: compute how many errors fall into each category and what share of total error it represents.
  5. Prioritize: weigh each category’s size against the estimated cost of fixing it.

The counting step yields the performance ceiling for each fix. If only 4% of errors come from blurry images, even a perfect blur fix cannot reduce total error by more than that share. This ceiling analysis keeps teams from spending weeks on causes that barely matter.

Common error categories and slicing dimensions

Useful error categories describe a cause or a data condition, not just a symptom. A taxonomy usually mixes data problems, model limitations and evaluation artifacts, and it differs by task type.

CategoryExampleUsual remedy
Label noiseReference label is itself wrongRelabel, measure inter-annotator agreement
Ambiguous inputsTwo answers are both defensibleClarify guidelines or accept multiple references
Underrepresented sliceRare dialect, rare product classCollect or augment targeted data
Distribution shiftNew camera, new user populationRetrain on recent data, monitor drift
Capability gapMulti-step reasoning failsLarger model, better training data, tools
HallucinationFluent but unsupported claimRetrieval grounding, abstention training
Format or instruction errorsIgnores required output schemaPrompt changes, constrained decoding
Typical error categories across model types

Beyond categories, analysts slice the evaluation data by metadata such as input length, language, source, time period or user segment. Slice-level metrics often reveal that a model with strong overall accuracy performs poorly on a small but important subgroup, a pattern that aggregate scores hide.

Error analysis answers a diagnostic question, whereas most neighboring evaluation methods answer a measurement question. The table shows how it relates to other techniques in the Model evaluation topic.

MethodQuestion it answersTypical output
Error analysisWhy and where does the model failRanked failure categories with examples
Confusion matrixWhich classes are confused with whichClass-by-class count table
Ablation studyWhich component contributes to performanceScore change per removed component
AI benchmarkHow does the model compare with othersStandardized score
Human evaluationHow do people judge output qualityRatings or win rates
How error analysis differs from neighboring techniques

These methods work best together. A confusion matrix often points to where to look, and error analysis then reads the confused cases to find the reason. An ablation study tests whether a proposed fix actually addresses the category that error analysis identified.

Error analysis for large language models

For generative language models, error analysis requires extra care because outputs are open-ended and a single response can fail in several ways at once. Analysts typically review full conversation traces, including retrieved documents and tool calls, rather than only the final answer.

Teams increasingly use LLM-as-a-judge to tag failures at scale, but the tags must be validated against human labels on a sample first. A judge model that agrees with humans on only 70% of cases will distort category counts, so measured agreement belongs in the analysis report.

SituationRecommended approachReason
Early prototypeManual review of 50 to 100 tracesTaxonomy is still unknown
Stable taxonomyAutomated tagging with a judge modelScales to thousands of cases
Retrieval pipelineSeparate retrieval errors from generation errorsFixes differ for each stage
Agent workflowLocate the first failing step in the traceLater failures often cascade
Practical choices in language model error analysis

Limitations and common mistakes

Error analysis is only as reliable as the sample and the taxonomy behind it. Several recurring mistakes weaken its conclusions.

  • Analyzing the test set: repeated inspection leaks information and turns the test set into a development set, inflating final scores.
  • Too few examples: with 30 errors, a category of 3 cases has a very wide uncertainty range, so rankings may be noise.
  • Vague categories: tags such as “model confused” describe symptoms and suggest no concrete action.
  • Ignoring correct cases: some causes also appear among successes, which means they may not drive failure at all.
  • One-time effort: failure profiles change after each model update, so the analysis must be repeated every iteration.

A further limitation is analyst bias. Reviewers tend to notice the failures they expect, so writing the taxonomy before reading examples, or having two reviewers tag independently, produces more trustworthy counts.

In everyday language

Error analysis is like a teacher going through a stack of graded exams and sorting every wrong answer into piles: careless arithmetic, misunderstood questions, topics never studied. Counting the piles shows where extra practice will help most.

For an AI model, the idea is the same. Instead of only knowing that the model is right 90% of the time, the team reads the wrong 10% and learns what kinds of mistakes it makes, so the next improvement targets the biggest problem first.

Seen in the real world

A support team uses a text classifier to route customer emails into twelve categories, and validation accuracy has stalled at 91%. Instead of trying a larger model, an engineer samples 200 misrouted emails and tags each one.

The counts are revealing: 38% of errors involve emails that mention two issues at once, 27% come from inconsistent reference labels between two similar categories, 15% are non-English messages and the rest are scattered. The team rewrites labeling guidelines, merges the two overlapping categories and adds a multi-label option for emails with two issues.

After retraining on the cleaned data, accuracy rises to about 95%, without any change to the model architecture. The non-English slice is scheduled for the next iteration, with targeted data collection as the planned fix.