Group fairness is a family of formal criteria that require a machine learning model to produce statistically comparable outcomes or error rates across groups defined by a protected attribute such as sex, ethnicity or age. In AI ethics, it is the most widely used way to turn the abstract goal of non-discrimination into measurable conditions that can be tested on data.
Group fairness compares aggregate statistics rather than individual cases. A model satisfies a given criterion when a chosen quantity, for example the rate of positive decisions or the false negative rate, is equal or nearly equal between groups. Because several of these quantities cannot all be equalized at once, choosing a criterion is a normative decision as much as a technical one.
How group fairness is measured
Group fairness is measured by computing a performance or outcome statistic separately for each group and comparing the results. The comparison is reported either as a difference, where zero means parity, or as a ratio, where one means parity.
Every measurement starts from the confusion matrix of each group: true positives, false positives, true negatives and false negatives. From these counts analysts derive selection rates, true positive rates, false positive rates and precision, then check which of them differ. The choice of statistic encodes a view of what kind of harm matters in the application.
The table below lists the main group fairness criteria and the statistic each one equalizes.
| Criterion | Quantity equalized across groups | Depends on true labels |
|---|---|---|
| Demographic parity | Rate of positive predictions | No |
| Equal opportunity | True positive rate | Yes |
| Equalized odds | True and false positive rates | Yes |
| Predictive parity | Precision of positive predictions | Yes |
| Calibration within groups | Observed rate at each score level | Yes |
| Accuracy parity | Overall accuracy | Yes |
Independence, separation and sufficiency
Most group fairness criteria fall into three families known as independence, separation and sufficiency. The families describe which statistical relationship among the prediction, the true outcome and the protected attribute is required.
- Independence requires the prediction to be statistically independent of the group; demographic parity is the standard example.
- Separation requires independence of the group conditional on the true outcome; equalized odds and equal opportunity belong here.
- Sufficiency requires the true outcome to be independent of the group conditional on the prediction; calibration and predictive parity belong here.
A central result in the field, the impossibility theorem of fairness, shows that separation and sufficiency cannot both hold when base rates differ between groups, unless the classifier is perfect. Independence also conflicts with the other two in the same conditions. This finding explains why disputes such as the debate over recidivism risk scores were not resolved by better engineering: each side relied on a different, internally consistent criterion.
Group fairness compared with other fairness notions
Group fairness differs from other notions mainly in its unit of analysis. It evaluates statistics over populations, whereas individual and counterfactual notions evaluate how each person is treated.
The following comparison places group fairness among the neighboring concepts of algorithmic fairness.
| Concept | Core requirement | Main weakness |
|---|---|---|
| Group fairness | Equal statistics across protected groups | Can hide unfair treatment of individuals |
| Individual fairness | Similar people receive similar decisions | Requires an agreed similarity measure |
| Counterfactual fairness | Decision unchanged if the attribute were different | Needs a trusted causal model |
| Fairness through unawareness | Protected attribute is not used as input | Fails when proxies carry the same signal |
| Disparate impact | Selection ratio not far below parity | Legal test rather than full criterion |
A known weakness of group fairness is fairness gerrymandering: a model can satisfy parity for each attribute separately while discriminating against an intersectional subgroup, such as older women. Subgroup and multicalibration methods extend the idea to many overlapping groups at once.
Choosing and applying a criterion
Choosing a group fairness criterion means matching the statistic to the harm the decision can cause and to the reliability of the labels. No criterion is correct in general, so the choice should be documented and justified in an algorithmic impact assessment.
| Situation | Suitable criterion | Reason |
|---|---|---|
| Labels reflect past discrimination | Demographic parity | Avoids reproducing biased historical outcomes |
| Missing a qualified person is the main harm | Equal opportunity | Equalizes access for those who merit it |
| Both false alarms and misses harm people | Equalized odds | Balances both error types across groups |
| Scores guide human judgment directly | Calibration within groups | Same score means the same risk for everyone |
| Hiring screens under employment law | Four-fifths rule check | Common regulatory benchmark for adverse impact |
Once a criterion is chosen, bias mitigation can be applied at three points: pre-processing the training data through reweighting or resampling, in-processing with a fairness constraint added to the training objective, or post-processing with group-specific decision thresholds. Post-processing is the simplest to audit, but it requires access to the protected attribute at decision time, which some jurisdictions restrict.
Limitations and common mistakes
The main limitation of group fairness is that statistical parity is not the same as justice. A metric can be met while the underlying data, the target variable or the use of the decision remain unfair.
- Biased labels: when the target records past decisions, criteria based on true labels can certify discrimination as accurate.
- Small groups: rates estimated on a few hundred cases carry wide confidence intervals, so apparent gaps may be noise.
- Missing attributes: many organizations do not collect protected data, which makes measurement depend on uncertain inference.
- Leveling down: parity can be reached by worsening outcomes for the better-served group, which helps no one.
- One-time audits: populations shift, so fairness must be monitored after deployment, not only tested before release.
Group fairness also interacts with the fairness-accuracy trade-off. Imposing a constraint usually reduces measured accuracy slightly, but when labels are biased the measured accuracy itself is misleading, so the loss may be smaller than it appears.
The simple explanation
Group fairness asks a simple question: does an automated system treat different groups of people about equally overall? For a loan model, that could mean checking whether men and women are approved at similar rates, or whether qualified applicants in each group are rejected equally often.
The difficulty is that equal can mean several different things, and they often cannot all be true together. Deciding which kind of equality matters is a choice about values, not only about mathematics.
How it is used in practice
A regional bank builds a model that recommends approving or declining small business loans. Before launch, its model risk team measures group fairness across applicant sex and age bands on a holdout set of 40,000 past applications with known repayment outcomes.
The approval rate is similar for all groups, but the analysis shows that creditworthy applicants over 60 are declined more often: their true positive rate is 9 percentage points lower than that of younger applicants. The team traces the gap to a feature measuring length of digital banking history, which acts as an age proxy.
After removing the feature and adjusting thresholds, the gap falls to under 2 points with a negligible drop in overall accuracy. The bank documents the chosen criterion, equal opportunity, in its impact assessment and schedules a quarterly fairness review.