Counterfactual explanation

On this page 7

A counterfactual explanation describes the smallest change to an input that would have led a machine learning model to a different outcome, in the form “if X had been different, the decision would have been Y”. In AI ethics, it is valued because it explains an individual decision without revealing the model’s internals and tells the affected person what could realistically be done differently.

A typical example is a declined loan: “If your annual income had been 4,000 dollars higher and your credit card balance 1,500 dollars lower, the application would have been approved.” The explanation is local (it concerns one decision), contrastive (it compares the actual outcome with a desired one) and model-agnostic in most implementations.

How a counterfactual explanation is computed

A counterfactual explanation is computed by searching for a nearby input that the model classifies differently. The original instance x receives an unwanted prediction, and the algorithm looks for a modified instance x’ that receives the desired prediction while staying as close to x as possible.

The classic formulation, introduced by Wachter, Mittelstadt and Russell in 2017, treats this as an optimization problem: minimize a loss that combines the distance between the model output and the target outcome with a distance penalty between x and x’. Later methods add further terms so that the result is not only close but also plausible and actionable.

The table below lists the properties that most modern methods try to balance.

PropertyMeaningWhy it matters
ValidityThe changed input truly flips the predictionOtherwise the explanation is false
ProximityChanges are as small as possibleKeeps the advice realistic
SparsityFew features changeEasier for people to understand
PlausibilityResult lies in a realistic data regionAvoids impossible combinations
ActionabilityOnly mutable features changeGives advice a person can follow
DiversitySeveral distinct alternatives are offeredLets the person choose a path
Desired properties of a counterfactual

Main generation methods

Counterfactual generation methods differ mainly in how they search and what access they need to the model. Gradient-based methods require a differentiable model, while search-based and genetic methods work with any black-box model that can be queried.

Method familyModel accessTypical strengthTypical weakness
Gradient-based optimizationGradients requiredFast and precise for neural networksCan produce implausible points
Diverse counterfactuals (DiCE)Gradients or queriesSeveral alternatives at onceTuning of diversity weights
Genetic and heuristic searchQueries onlyWorks with any model typeMany model calls needed
Prototype-guided searchGradients or queriesResults resemble real dataNeeds a representative dataset
Integer programming for recourseLinear or tree modelsGuarantees on optimalityLimited to simple model classes
Common families of counterfactual methods

Distance is usually measured with an L1 norm scaled by feature spread, such as the median absolute deviation, because it favors sparse changes. Categorical features require special handling, since a change from “renter” to “homeowner” has no natural numeric size.

Counterfactual explanations compared with feature attribution

Counterfactual explanations answer “what would need to change”, whereas feature attribution methods such as SHAP and LIME answer “which features contributed most”. Both are forms of post-hoc explanation, but they serve different audiences and purposes.

AspectCounterfactual explanationSHAP or LIME
Question answeredWhat change flips the outcomeHow much each feature contributed
Output formAlternative input valuesImportance scores per feature
Main audienceAffected individualsData scientists and auditors
Supports recourseYes, directlyOnly indirectly
UniquenessMany valid answers may existUsually one score set per method
Counterfactuals versus attribution-based explanations

The term is also distinct from counterfactual fairness, a fairness criterion that asks whether a prediction would stay the same if a protected attribute had been different in a causal model. Counterfactual explanations describe a decision; counterfactual fairness evaluates whether the decision process is fair.

Role in accountability and regulation

Counterfactual explanations support algorithmic recourse, the ability of a person to obtain a better outcome by acting on their circumstances. This links them closely to contestability and AI accountability, because a person who understands the decisive factors can challenge an error or take targeted steps.

Legal discussions often connect counterfactuals with the GDPR provisions on automated decision-making and with adverse action notices in United States consumer credit, which require lenders to state the principal reasons for a denial. Counterfactuals do not automatically satisfy these duties, but they offer a format that is understandable to non-experts and does not disclose trade secrets.

Limitations and common pitfalls

The most serious limitation is the Rashomon effect: many different counterfactuals can be equally valid, and presenting only one may hide other paths or reflect the designer’s choices. Selecting which one to show is itself an ethical decision with consequences for the person.

The following table lists frequent problems in practice, their causes and the usual remedies.

ProblemTypical causeMitigation
Advice to change age or ethnicityNo mutability constraintsFreeze immutable features
Unrealistic combinationsDistance ignores data distributionAdd plausibility or density terms
Ignored side effectsFeatures treated as independentUse a causal model of features
Recourse no longer validModel retrained after adviceTest robustness across model versions
Gaming of the systemAdvice targets proxies, not meritPrefer causally meaningful features
Pitfalls when deploying counterfactual explanations

Counterfactuals can also leak information about the model, since repeated queries with diverse explanations allow an adversary to approximate the decision boundary. Organizations therefore balance transparency against model extraction risk.

The short version

A counterfactual explanation answers a simple question after an AI system says no: what would have made it say yes? There is no long technical report. The person gets one or a few concrete changes. Examples are a higher income or less debt.

It works like a teacher who returns a failed test with a short note. The note says that five more correct answers would have meant a pass. It does not explain how the grading works inside. But it shows where the line was and what to do next time.

Where it makes a difference

A regional bank uses a gradient boosting model to screen small business loan applications. A café owner applies for a 50,000-dollar loan and is declined. Instead of a generic letter, the bank’s system generates three diverse counterfactual explanations, restricted to features the applicant can actually change and checked against realistic data ranges.

One option shows approval if existing credit line usage fell from 85 percent to below 40 percent. Another shows approval for a reduced amount of 35,000 dollars. A third requires six more months of documented revenue. The owner chooses to request the smaller loan, which is approved, and plans to reapply later.

Internally, the bank’s model risk team also reviews aggregated counterfactuals across all declined applicants and discovers that postal code changes often flipped decisions, a sign of possible proxy discrimination. The feature is removed after an audit, showing how counterfactuals serve both individual recourse and fairness oversight.