Predictive parity

On this page 7

Predictive parity is a group fairness criterion that requires a classifier’s positive predictive value to be equal across protected groups: among all people the model flags as positive, the share who are truly positive must be the same for each group. In AI ethics, it expresses the idea that a positive prediction should mean the same thing regardless of who receives it.

Formally, predictive parity holds when the probability that the true outcome is positive, given a positive prediction, does not depend on the protected attribute. It belongs to the family of sufficiency-based metrics, alongside calibration within groups, and it often conflicts with error-rate criteria such as equalized odds.

How predictive parity is measured

Predictive parity is measured by computing precision, also called positive predictive value (PPV), separately for each group and comparing the results. PPV equals true positives divided by all predicted positives, so it answers the question: when the model says yes, how often is it right?

In practice, auditors report either the difference or the ratio between group PPVs. Exact equality is rare with real data, so teams set a tolerance band, for example a gap below a few percentage points, and test whether the difference is statistically meaningful given the sample sizes.

The table below shows a worked comparison for a hypothetical loan default model with two applicant groups.

QuantityGroup AGroup B
Applicants flagged as high risk400200
Flagged applicants who defaulted240120
Positive predictive value0.600.60
Base rate of default0.300.15
Predictive parity satisfiedYesYes
Predictive parity check for a hypothetical default model

The example shows that equal PPV can coexist with very different base rates. A related quantity, the negative predictive value, is sometimes required to match as well; requiring both is a stricter condition sometimes called conditional use accuracy equality.

Predictive parity among other fairness metrics

Predictive parity differs from other group fairness metrics in which conditional probability it holds constant. Each criterion conditions on a different quantity, which is why they encode different moral intuitions and why they can disagree on the same model.

CriterionWhat must be equalConditioned onFamily
Predictive parityPrecision (PPV)Positive predictionSufficiency
Calibration within groupsObserved rate at each scoreRisk score valueSufficiency
Demographic parityPositive prediction rateGroup onlyIndependence
Equal opportunityTrue positive rateTrue positive outcomeSeparation
Equalized oddsTrue and false positive ratesTrue outcomeSeparation
Predictive parity compared with neighboring group criteria

Predictive parity takes the perspective of the decision-maker, who wants a flag to carry equal evidential weight for everyone. Error-rate criteria such as equalized odds take the perspective of the affected individual, who wants equal chances of being wrongly flagged or wrongly cleared.

The impossibility result and the COMPAS debate

A well-known impossibility theorem shows that predictive parity and equal error rates cannot generally hold together. When base rates differ between groups and the classifier is imperfect, equal PPV forces unequal false positive or false negative rates, and the reverse is also true.

This result, published independently by Alexandra Chouldechova and by Jon Kleinberg and colleagues in 2016 and 2017, arose from the dispute over the COMPAS recidivism risk tool. Journalists showed that Black defendants faced substantially higher false positive rates, while the vendor replied that the scores were equally predictive across races, which is essentially a predictive parity argument.

ConditionsPredictive parity possibleEqual error rates possible
Equal base rates, imperfect modelYes, togetherYes, together
Different base rates, perfect modelYes, togetherYes, together
Different base rates, imperfect modelYes, aloneYes, alone, not with PPV parity
What can and cannot hold at the same time

Both sides in the COMPAS dispute were therefore arithmetically correct. The disagreement was about which notion of fairness should govern criminal justice decisions, a normative question that statistics alone cannot settle.

When to choose predictive parity

Predictive parity is most appropriate when a positive prediction triggers a costly action and the main concern is that the action should be equally justified for every group. It is weaker when the harm lies mainly in being wrongly flagged, because it does not equalize false positive rates.

SituationIs predictive parity a good fitReason
Fraud alerts sent to investigatorsOften yesEach alert should justify equal effort
Medical screening for follow-up testsPartlyMissed cases also matter, check recall
Pretrial detention recommendationsContestedFalse positives deprive people of liberty
Hiring shortlist generationUsually not aloneUnequal selection rates raise disparate impact
Decision guide for selecting predictive parity
  • Report several metrics together so that trade-offs are visible rather than hidden behind one number.
  • Document the chosen criterion and its rationale in an AI audit or algorithmic impact assessment.
  • Involve affected stakeholders when the choice between metrics carries moral weight.

Limitations and common misconceptions

The most common misconception is that satisfying predictive parity makes a model fair. It only guarantees equal precision for positive predictions and says nothing about who is missed, who is wrongly flagged or whether the labels themselves are reliable.

Predictive parity also inherits label bias. If historical outcomes reflect unequal policing or unequal access to credit, then equal PPV against those labels can legitimize measurement bias and historical bias instead of correcting them. Small groups produce unstable PPV estimates, and the metric can shift when deployment thresholds or populations change.

Finally, enforcing predictive parity by setting group-specific thresholds may itself raise legal questions where the explicit use of a protected attribute in decisions is restricted, so technical and legal review should proceed together.

In plain words

Predictive parity asks a simple question: when the system raises a red flag, is that flag equally trustworthy for everyone? If a warning is right 60 percent of the time for one group, it should also be right 60 percent of the time for another.

Think of a smoke alarm that sounds in two kitchens. Predictive parity checks that an alarm means real smoke equally often in both. It does not check whether one kitchen gets far more false alarms in total or whether real fires are missed, so it is only one piece of fairness.

A real-world example

A regional health insurer uses a model to flag claims for manual fraud review. An internal audit team splits the past year of flagged claims by the age group of the policyholder. For members under 65, 312 of 520 flagged claims proved to be fraudulent, a PPV of 0.60; for members 65 and older, only 96 of 240 flagged claims were fraudulent, a PPV of 0.40.

The gap means older members were flagged on weaker evidence, causing payment delays without justification. The team raised the review threshold for that segment’s score band after confirming that the age-specific adjustment was legally permissible, and the next quarter showed PPV of 0.58 and 0.57. The audit report also noted a remaining difference in recall, so the insurer documented the trade-off explicitly rather than claiming the model was fully fair.