Fairness metric

On this page 7

A fairness metric is a quantitative measure of whether a machine learning model treats groups or individuals equitably, typically by comparing prediction rates, error rates or calibration across values of a protected attribute such as sex, age or ethnicity. In AI ethics, fairness metrics turn abstract principles of non-discrimination into numbers that can be tested, monitored and audited.

No single fairness metric captures fairness as a whole. Each one encodes a specific normative definition, and several common definitions are mathematically incompatible when base rates differ between groups, so choosing a metric is as much an ethical and legal decision as a technical one in artificial intelligence.

How a fairness metric is calculated

Most group fairness metrics start from the confusion matrix computed separately for each group. For a binary classifier, the evaluator counts true positives, false positives, true negatives and false negatives within each group, then derives rates such as the selection rate, the true positive rate and the false positive rate.

The metric is then expressed as a difference or a ratio between groups. A difference of 0 or a ratio of 1 means perfect parity on that criterion. In practice, organizations define a tolerance band, for example a difference below 0.05 or a ratio between 0.8 and 1.25, because exact equality is rarely achievable with finite data.

The table below shows the building blocks from which most group metrics are composed.

RateFormulaQuestion it answers
Selection ratePredicted positives / group sizeHow often does the group receive the favorable outcome
True positive rateTP / (TP + FN)How many qualified members are correctly approved
False positive rateFP / (FP + TN)How many unqualified members are wrongly approved
Positive predictive valueTP / (TP + FP)How reliable is a positive prediction
False negative rateFN / (FN + TP)How many qualified members are wrongly rejected
Rates used to compute group fairness metrics

Main families of fairness metrics

Fairness metrics fall into three broad families: group fairness, which compares statistics across groups; individual fairness, which requires similar individuals to receive similar outcomes; and counterfactual fairness, which asks whether a prediction would change if the protected attribute were different while everything causally unrelated stayed the same.

Group metrics dominate in practice because they are easy to compute and explain. Individual and counterfactual metrics are closer to intuitive notions of justice but require a defensible similarity measure or a causal model of the data, which is often contested.

MetricCondition for fairnessFamily
Demographic parityEqual selection rates across groupsGroup, independence
Disparate impact ratioRatio of selection rates of at least 0.8Group, independence
Equal opportunityEqual true positive ratesGroup, separation
Equalized oddsEqual true and false positive ratesGroup, separation
Predictive parityEqual positive predictive valuesGroup, sufficiency
Calibration within groupsSame score means same probability in every groupGroup, sufficiency
Individual fairnessSimilar individuals receive similar scoresIndividual
Counterfactual fairnessPrediction unchanged in counterfactual worldCausal
Common fairness metrics and what they require

The labels independence, separation and sufficiency describe the statistical relationship that each group metric demands between the prediction, the true outcome and the protected attribute. They provide a compact way to see which metrics conflict.

Why fairness metrics conflict with each other

Several well-known impossibility results show that, when the base rate of the positive outcome differs between groups, a classifier that is not perfect cannot satisfy calibration and equal error rates at the same time. The debate over recidivism risk scores made this concrete: one analysis showed unequal false positive rates, while the developer showed equal calibration, and both statements were true.

The practical consequence is that every metric choice favors some stakeholders over others. Demographic parity can require approving less qualified applicants or rejecting qualified ones, while predictive parity can preserve historical disparities embedded in the labels. This tension is also why practitioners discuss a fairness-accuracy trade-off, although the size of that trade-off is often smaller than assumed once data quality improves.

Decision contextOften preferred metricReason
Hiring shortlistsDisparate impact ratio, demographic parityAligned with employment discrimination practice
Medical screeningEqual opportunityMissed diagnoses are the costliest error
Credit scoringCalibration, equalized oddsScores must mean the same risk for everyone
Fraud detectionEqual false positive ratesFalse accusations harm innocent customers
Content rankingExposure parityVisibility is the allocated resource
Choosing a fairness metric by decision context

Applying fairness metrics in practice

A sound process begins by defining the harm: who could be disadvantaged, by which error type and with what consequence. Only then does the team select one primary metric and one or two secondary metrics, set explicit thresholds, and document the reasoning in model documentation or an algorithmic impact assessment.

Measurement requires reliable group labels, which are often missing or legally restricted. Teams then rely on self-reported data collected for auditing only, on inferred attributes with known error, or on external audits. Results should be reported with confidence intervals, because small groups produce unstable rates that can show disparities caused purely by sampling noise.

Fairness metrics should also be evaluated for intersectional subgroups, such as older women or young men from a specific region, since a model can appear fair on each attribute separately while disadvantaging a combination. Open-source toolkits automate these calculations, but they do not decide which metric is appropriate.

Limitations and common misuses

A fairness metric measures one statistical property of one model on one dataset. It cannot detect harm that lies outside the measured outcome, and it inherits every flaw of the labels. If historical decisions were biased, metrics based on true outcomes treat that bias as ground truth, a problem known as label bias or historical bias.

MistakeConsequenceBetter practice
Reporting a single metricHidden disparities on other criteriaReport several metrics side by side
Ignoring sample sizeSpurious or missed disparitiesUse confidence intervals and minimum group sizes
Dropping the protected attributeProxy discrimination remains unmeasuredKeep the attribute for auditing
Measuring once before launchDrift creates new disparities laterMonitor metrics continuously
Treating parity as proof of fairnessFalse assurance and ethics washingCombine metrics with qualitative review
Frequent mistakes with fairness metrics

Fairness metrics are therefore necessary but not sufficient. They belong inside a broader governance process that includes stakeholder input, legal review, contestability mechanisms and periodic audits of real-world outcomes.

Explained without jargon

A fairness metric is a scorecard for even treatment. It checks whether an automated system gives good outcomes, or makes mistakes, more often for one group of people than for another.

Imagine a referee who calls more fouls against one team. You could count fouls per team, or count only the wrong calls per team, and the two counts can tell different stories. Fairness metrics work the same way: each one counts something different, so people must agree on which count matters most before they can say a system is fair.

A typical use case

A mid-sized bank builds a model that recommends approval or rejection for small personal loans. Before deployment, the model risk team measures fairness by age group and sex using a held-out test set of about 40,000 past applications.

The model shows a disparate impact ratio of 0.76 for applicants over 60, below the 0.8 threshold the bank set as its internal limit. Further analysis reveals that the false negative rate for this group is nine percentage points higher, meaning that creditworthy older applicants are rejected more often. The cause is a feature measuring the length of recent digital banking activity, which acts as a proxy for age.

The team removes that feature, retrains the model and recalibrates thresholds. The ratio rises to 0.88, the false negative gap falls to two points and overall accuracy drops by less than one percentage point. The bank documents the analysis and schedules quarterly fairness monitoring after launch.