Counterfactual fairness

On this page 7

Counterfactual fairness is a causal definition of fairness under which a prediction is fair to an individual if it would have been the same in a counterfactual world where that person’s protected attribute, such as race or sex, had been different, with everything not caused by that attribute held fixed. In AI ethics, it is one of the main attempts to move fairness from statistical patterns toward explicit reasoning about cause and effect.

The concept was formalized in 2017 by Kusner, Loftus, Russell and Silva, building on structural causal models developed by Judea Pearl. It answers a question that group metrics cannot: whether a specific decision depends on the protected attribute through causal pathways that society considers illegitimate.

How counterfactual fairness is defined

Counterfactual fairness is defined with respect to a structural causal model, a set of equations and a directed graph that describe how variables cause one another. The protected attribute A, observed features X and unobserved background factors U are all part of this model.

A predictor is counterfactually fair if, for any individual with observed values, the distribution of the prediction is identical whether A takes its actual value or is intervened on to take another value. Formally, the prediction under the intervention do(A = a) must equal the prediction under do(A = a’) for every a’, given the same background factors U.

Computing this requires three classic steps of counterfactual inference, summarized below.

StepWhat happensPurpose
AbductionInfer background factors U from observed dataFix what is specific to the individual
ActionSet the protected attribute to another valueCreate the counterfactual scenario
PredictionRecompute descendants of A and the model outputCompare outcomes across both worlds
The three steps of counterfactual inference

The key insight is that changing A also changes every feature causally downstream of A. Removing the protected attribute from the inputs is therefore not enough, because proxies such as postal code or school attended may carry its influence.

Building a counterfactually fair model

A model achieves counterfactual fairness by using only information that is not a causal descendant of the protected attribute. The original paper describes three levels of modeling, which trade stronger assumptions for more usable information.

LevelApproachAssumptions required
Level 1Use only observed non-descendants of ACausal graph only, no equations
Level 2Infer latent variables not caused by AGraph plus a latent variable model
Level 3Model features as functions of A plus residual noiseFully specified additive equations
Modeling levels for counterfactually fair prediction

Level 1 is often too restrictive, since few useful variables are entirely independent of protected attributes. Levels 2 and 3 recover more predictive power by extracting the part of each feature that A does not explain, but they depend on correct model specification. If the assumed equations are wrong, the resulting predictor may look fair on paper while remaining unfair in reality.

A related approach called path-specific counterfactual fairness relaxes the definition. It blocks only the causal paths judged unfair, for example the path from gender to hiring through name, while allowing paths judged legitimate, such as gender influencing a chosen field of study that is genuinely relevant to the job.

Counterfactual fairness compared with other fairness criteria

Counterfactual fairness differs from most criteria in AI ethics because it is individual-level and causal, while popular metrics such as demographic parity or equalized odds are group-level and statistical. The comparison below shows what each criterion checks.

CriterionLevelWhat it requiresNeeds a causal model
Counterfactual fairnessIndividualSame prediction if A had differedYes
Fairness through unawarenessIndividualA is not used as an inputNo
Individual fairnessIndividualSimilar people get similar outcomesNo, needs a similarity metric
Demographic parityGroupEqual positive rates across groupsNo
Equalized oddsGroupEqual error rates across groupsNo
Counterfactual fairness and related fairness criteria

Counterfactual fairness is strictly stronger than fairness through unawareness, which only drops A and ignores proxies. It is also distinct from a counterfactual explanation, which tells a person what change in their features would alter a decision and does not by itself test fairness.

When counterfactual fairness is useful

Counterfactual fairness is most valuable when decision makers can agree on a plausible causal graph and need to justify individual decisions rather than aggregate statistics. Typical settings include lending, admissions, hiring and insurance pricing.

SituationSuitabilityReason
Credit scoring with known proxiesHighPaths through postal code can be modeled
University admissionsModerateSome paths are contested, needs path-specific view
Medical risk predictionLimitedBiological effects of sex may be legitimate
Image or text modelsLowCausal graphs over raw pixels or tokens are unclear
Situations where counterfactual reasoning helps or struggles

In practice, teams often use counterfactual fairness as a diagnostic tool rather than a hard constraint, for example by flipping a protected attribute in a simulation and measuring how many decisions change. Such attribute-flip tests are cheaper, although they only approximate the full definition because they rarely update downstream features.

Limitations and criticisms

The central limitation is that counterfactual fairness is only as valid as the causal model behind it, and causal graphs for social phenomena are usually uncertain and contested.

  • Untestable assumptions: counterfactuals about the same person cannot be observed, so the equations cannot be fully validated from data.
  • Questions about manipulating identity: some scholars argue that intervening on race or sex is conceptually problematic, because these attributes are bundles of social meaning rather than switchable variables.
  • Fairness-accuracy trade-off: removing all descendant information can lower predictive performance, sometimes substantially.
  • Model disagreement: different reasonable causal graphs can yield different verdicts on the same predictor, so the choice of graph becomes a value-laden decision.
  • Limited legal mapping: anti-discrimination law rarely uses causal language directly, so translating a counterfactual guarantee into compliance requires careful interpretation.

Despite these limits, the concept has had lasting influence, because it forces practitioners to state their assumptions explicitly and to distinguish legitimate from illegitimate influences instead of relying on a single statistical number.

The short version

Counterfactual fairness asks one simple question about each decision. Imagine the same person, but of a different race or sex. Would the answer stay the same?

If the answer would change, the system treats that person differently because of who they are. The tricky part is deciding what stays unchanged. Some details, like where someone grew up, may be shaped by their group. The idea works only with agreed causes.

Where it makes a difference

A regional lender reviews a credit model that does not use race as an input. An audit team builds a causal graph linking race to neighborhood, neighborhood to postal code and property values, and those to the model’s inputs, alongside variables such as income history and repayment record.

Using the graph, the team estimates each applicant’s counterfactual score if their race had been different. For about 7 percent of applicants, the decision flips, almost entirely through the postal code feature. The team replaces postal code with a residual measure that removes the part explained by race and retrains the model.

The revised model loses about one percentage point of accuracy, but counterfactual flips fall below 1 percent. The lender documents the causal assumptions, so regulators and internal reviewers can challenge the graph itself rather than only the final scores.