A confusion matrix is a table that cross-tabulates the true classes of examples against the classes a model predicted, so that every correct prediction and every type of error is counted separately. In model evaluation for artificial intelligence and machine learning, it is the foundation from which most classification metrics are calculated, including accuracy, precision, recall and the F1 score.
Its value lies in showing what kind of mistakes a classifier makes, not just how many. Two models with the same accuracy can have completely different confusion matrices, and in practice the type of error often matters more than the total, for example when a missed disease is far more costly than a false alarm.
Structure of a confusion matrix
For a binary classifier, the confusion matrix has four cells. By the most common convention, rows represent the actual class and columns represent the predicted class, although some libraries and textbooks transpose this, so always check the axis labels before reading it.
| Cell | Actual class | Predicted class | Meaning |
|---|---|---|---|
| True positive (TP) | Positive | Positive | Correctly detected case |
| False negative (FN) | Positive | Negative | Missed case, a type II error |
| False positive (FP) | Negative | Positive | False alarm, a type I error |
| True negative (TN) | Negative | Negative | Correctly rejected case |
For a classifier with k classes, the matrix becomes k × k. The diagonal holds correct predictions, and each off-diagonal cell shows how often one specific class was mistaken for another. This makes systematic confusions visible at a glance, such as a model that repeatedly labels wolves as huskies or invoices as receipts.
The matrix depends on a decision threshold. A model that outputs a probability must convert it into a class, typically at 0.5 by default; moving the threshold shifts examples between cells and therefore changes every derived metric.
Metrics derived from the confusion matrix
Nearly every classification metric is a ratio of confusion matrix cells. Understanding which cells each metric uses explains what the metric rewards and what it ignores.
| Metric | Formula | Question it answers |
|---|---|---|
| Accuracy | (TP + TN) / all | How often is the model right overall |
| Precision | TP / (TP + FP) | How many predicted positives are real |
| Recall or sensitivity | TP / (TP + FN) | How many real positives are found |
| Specificity | TN / (TN + FP) | How many real negatives are correctly rejected |
| F1 score | 2 × precision × recall / (precision + recall) | Balance of precision and recall |
| False positive rate | FP / (FP + TN) | How often negatives trigger alarms |
| Matthews correlation coefficient | Uses all four cells, from -1 to 1 | Balanced quality on skewed classes |
The ROC curve plots recall against the false positive rate across all thresholds, and AUC summarizes it in one number. Each point on that curve corresponds to one confusion matrix, which shows that threshold-free metrics are summaries of many matrices rather than replacements for them.
Reading a multiclass confusion matrix
In multiclass problems, the confusion matrix reveals which pairs of classes the model cannot separate. Large off-diagonal values usually point to similar-looking classes, ambiguous labeling guidelines or missing features.
Per-class precision and recall are combined into a single score by averaging. The averaging method changes the result considerably when classes are imbalanced, so it must always be reported.
| Averaging method | How it works | When to use it |
|---|---|---|
| Macro average | Unweighted mean over classes | Every class matters equally, including rare ones |
| Weighted average | Mean weighted by class frequency | Overall performance reflecting class mix |
| Micro average | Pools all cells before computing | Overall instance-level performance |
Normalizing the matrix by row converts counts into per-class recall percentages and makes rare classes readable. Normalizing by column shows per-class precision. Raw counts remain useful to judge whether a percentage rests on five examples or five thousand.
Using the confusion matrix to make decisions
The confusion matrix connects model metrics to business and safety costs. Assigning a cost to each cell turns evaluation into a decision problem: choose the threshold that minimizes total expected cost, not the one that maximizes accuracy.
| Application | Costlier error | Metric to prioritize |
|---|---|---|
| Cancer screening | False negative, a missed tumor | Recall at acceptable precision |
| Spam filtering | False positive, a lost legitimate email | Precision |
| Fraud detection | False negative, approved fraud | Recall, with review capacity limits |
| Content moderation appeals | False positive, wrongly removed post | Precision on removal decisions |
| Manufacturing defect inspection | False negative, shipped defect | Recall |
Confusion matrices should be computed on a held-out evaluation dataset or through cross-validation, never on training data, because a model that overfits can produce an almost perfect diagonal on examples it has memorized.
Pitfalls and misconceptions
The most common pitfall is trusting accuracy on imbalanced data. If one percent of transactions are fraudulent, a model that never flags fraud reaches 99 percent accuracy while its confusion matrix shows zero true positives.
- Confusing the axis convention, which swaps precision and recall and leads to wrong conclusions.
- Reporting one matrix at an arbitrary threshold without stating that the threshold was chosen.
- Ignoring prevalence shift, since precision changes when the share of positives in production differs from the test set.
- Hiding small classes in weighted averages, which can mask a model that never predicts them.
- Comparing matrices from different test sets, which makes differences meaningless.
The idea in brief
Imagine a smoke detector tested on one hundred situations, some with real smoke and some without. A confusion matrix is simply a scorecard with four boxes: fires it caught, fires it missed, false alarms and quiet moments it correctly ignored.
Looking at the four boxes tells you much more than a single score. A detector that never misses a fire but often beeps at toast behaves very differently from one that is quiet but occasionally misses real smoke.
A practical scenario
A bank tests a new fraud detection model on 100,000 historical card transactions, of which 500 were fraudulent. At the default threshold of 0.5, the confusion matrix shows 300 true positives, 200 false negatives and 900 false positives. Accuracy is about 98.9 percent, which looks excellent, but recall is only 60 percent.
The fraud team estimates that each missed fraud costs far more than one manual review. Lowering the threshold to 0.2 produces 430 true positives and 2,600 false positives, raising recall to 86 percent while precision falls to about 14 percent. Because the team has capacity to review roughly 3,000 alerts per period, it adopts the lower threshold. The confusion matrix, not accuracy, drove the decision.