Matthews correlation coefficient

Also known as: MCC

On this page 7

The Matthews correlation coefficient (MCC) is a classification metric that measures the correlation between predicted and actual labels using all four cells of the confusion matrix: true positives, true negatives, false positives and false negatives. It ranges from -1 (total disagreement) through 0 (no better than chance) to +1 (perfect prediction).

In model evaluation for artificial intelligence and machine learning, MCC is valued because it remains informative on imbalanced datasets, where accuracy and even the F1 score can look excellent for a model that has learned very little. A high MCC requires the classifier to perform well on both the positive and the negative class at the same time.

How the Matthews correlation coefficient is calculated

MCC is computed directly from the confusion matrix with a single formula: MCC = (TP x TN - FP x FN) / sqrt((TP + FP)(TP + FN)(TN + FP)(TN + FN)). The numerator rewards correct predictions on both classes and penalizes both error types, while the denominator normalizes by the marginal totals so the result is bounded.

Mathematically, MCC is the Pearson correlation coefficient computed between two binary variables, the true label and the predicted label. It is also known in statistics as the phi coefficient. Because of this identity, MCC can be read the way any correlation is read: values near zero mean the predictions carry almost no information about the truth.

The following table shows how typical MCC values are usually interpreted in practice.

MCC rangeInterpretationTypical situation
+0.9 to +1.0Near-perfect agreementWell-separated classes, strong model
+0.5 to +0.9Strong to moderate agreementUseful production classifier
+0.1 to +0.5Weak agreementHard task or underfitting model
About 0No better than random guessingConstant predictor or broken pipeline
Below 0Systematic disagreementInverted labels or a severe bug
Interpreting MCC values

These bands are conventions, not universal thresholds. What counts as a good MCC depends on the domain and on the baseline achieved by simpler models.

Why MCC is robust to class imbalance

MCC is robust to imbalance because it cannot be maximized by favoring the majority class. Accuracy counts correct predictions regardless of class, and the F1 score ignores true negatives entirely, so both can reward degenerate behavior.

Consider a test set of 1,000 transactions containing 10 frauds. The comparison below shows how three common metrics react to the same weak classifiers.

Classifier behaviorAccuracyF1 scoreMCC
Predicts all negative0.9900.0000.000 (undefined, set to 0)
Flags 100 cases, catches 8 frauds0.9060.145About 0.23
Flags 12 cases, catches 8 frauds0.9940.727About 0.73
Metric behavior on a dataset with 1 percent positives

The all-negative model achieves 99 percent accuracy while being useless, and MCC correctly reports zero. The second model shows that a modest F1 can hide either a weak or a reasonable model, whereas MCC places it clearly in the weak agreement band. MCC also has symmetry: swapping which class is called positive leaves the value unchanged, which is not true of precision, recall or F1.

MCC compared with other evaluation metrics

MCC is one of several single-number summaries of a confusion matrix, and each answers a slightly different question. Choosing among them depends on which errors matter and whether the class balance of the test set reflects deployment.

MetricUses true negativesSensitive to imbalanceBest suited for
AccuracyYesHighly misleadingBalanced classes, equal error costs
F1 scoreNoModeratelyPositive-class retrieval tasks
Balanced accuracyYesRobustAverage recall across classes
Cohen’s kappaYesPartly robustAgreement beyond chance
AUCYes, across thresholdsCan be optimisticRanking quality independent of threshold
MCCYesRobustOne-threshold summary of all four outcomes
How MCC relates to neighboring metrics

Unlike AUC, MCC is evaluated at a single decision threshold, so it reflects the classifier as it would actually be deployed. Unlike balanced accuracy, it also accounts for precision on each class, which makes it stricter when false positives are numerous.

Multiclass MCC and practical use

MCC extends to multiclass classification through a generalization often attributed to Gorodkin, which computes the statistic from the full K by K confusion matrix. In that case the lower bound lies between -1 and 0 depending on the class distribution, while +1 still means perfect prediction. Common libraries such as scikit-learn implement this multiclass form under the same function name.

Practitioners typically use MCC in the situations summarized below.

SituationRecommendationReason
Severe class imbalanceReport MCC as a headline metricExposes majority-class shortcuts
Threshold selectionChoose the threshold that maximizes MCC on the validation setBalances both error types
Comparing models across datasetsReport MCC alongside the confusion matrixValues are comparable and bounded
Highly asymmetric error costsSupplement with cost-weighted metricsMCC weights both errors equally
Ranking or retrieval qualityPrefer AUC or precision-recall curvesMCC ignores score ordering
When to report MCC

Limitations and common mistakes

MCC is a strong default but not a complete evaluation. Its main weaknesses come from what a single correlation cannot express.

  • Undefined values: when any marginal total is zero, for example when the model never predicts the positive class, the denominator is zero; libraries usually return 0, which should be reported explicitly.
  • Equal error weighting: MCC treats false positives and false negatives symmetrically, which is wrong for screening tasks where a missed case costs far more.
  • Small-sample instability: with few positives, one prediction can move MCC substantially, so confidence intervals via bootstrapping are advisable.
  • Loss of detail: a single number hides whether errors are mostly false positives or false negatives; always publish the confusion matrix alongside it.
  • Prevalence dependence: MCC can change when the test set’s class ratio differs from deployment, so evaluation data should reflect realistic prevalence.

The short version

The Matthews correlation coefficient is a single score that tells whether a yes-or-no predictor is genuinely useful. It runs from minus one to plus one. Plus one means always right, zero means no better than a coin toss, and minus one means always wrong.

Its big advantage is that it cannot be fooled by lopsided data. If only one email in a hundred is spam, a filter that never flags anything is right 99 percent of the time, yet it is worthless. MCC gives that filter a score of zero, which is the honest answer.

Where it makes a difference

A hospital analytics team compares two models that predict which patients will develop sepsis within 48 hours. Only 3 percent of admissions in the test set develop sepsis. Model A reports 96.5 percent accuracy and Model B reports 94.0 percent accuracy, so an early review favors Model A.

When the team computes MCC, the picture reverses. Model A reaches an MCC of about 0.18, because it rarely flags anyone and misses most true cases. Model B reaches an MCC of about 0.52, catching most septic patients at the cost of more alerts. After reviewing both confusion matrices with clinicians, the team selects Model B and tunes its threshold on validation data to maximize MCC, then adds a cost-weighted review because a missed sepsis case is far more serious than a false alarm.