AI bias is a systematic error in an AI system that produces unfair or skewed outcomes for particular groups of people, kinds of input or situations. Unlike random errors, biased errors follow a pattern: they repeatedly favor or disadvantage the same groups, for example by approving fewer loans for one demographic or misrecognizing certain accents more often.
In AI safety and governance, bias matters because AI systems operate at scale. A single biased model can affect millions of decisions consistently, and its errors can be harder to notice and challenge than those of an individual person.
Where AI bias comes from
Bias can enter at every stage of the AI lifecycle, not only through training data. Identifying the source is the first step toward fixing it, because each source calls for a different remedy.
| Source | What happens | Example |
|---|---|---|
| Historical bias | Data reflects past unfair decisions | Hiring data favoring one gender |
| Representation bias | Some groups are underrepresented | Few images of darker skin tones |
| Measurement bias | Proxy variables measure unevenly | Arrests used as proxy for crime |
| Label bias | Annotators apply labels inconsistently | Dialects marked as offensive |
| Aggregation bias | One model fits distinct groups poorly | Medical thresholds set on one population |
| Deployment bias | Use differs from intended context | Tool used on a new population |
Models also amplify existing patterns. Because training rewards predicting the most common outcome, a mild imbalance in the data can become a stronger imbalance in the predictions. Removing protected attributes such as gender or ethnicity does not solve this, because proxy variables like postal code, name or purchase history often carry the same information.
Bias in generative and language models
Large language models and image generators learn from web-scale text and images, which contain the stereotypes and imbalances of the societies that produced them. Bias then appears as stereotyped associations, uneven quality across languages and dialects, or default depictions, such as generating mostly men when asked for an image of an engineer.
Fine-tuning with human feedback can reduce overt stereotypes but may introduce new preferences reflecting the views of the people who rated the outputs. Bias in generative systems is therefore measured with targeted prompt sets that vary only one attribute at a time and compare the results.
How AI bias is measured
Bias is measured by comparing a model’s behavior across groups using fairness metrics. There are many such metrics, and they formalize different ideas of what fair treatment means.
| Metric | Requires equal | Fits when |
|---|---|---|
| Demographic parity | Positive outcome rates | Access to an opportunity should be equal |
| Equal opportunity | True positive rates | Qualified people should be found equally |
| Equalized odds | True and false positive rates | Both benefits and harms should be equal |
| Predictive parity | Precision across groups | A positive prediction must mean the same |
| Calibration | Score meaning across groups | Risk scores guide human decisions |
A central and often misunderstood finding is that these metrics generally cannot all be satisfied at once when base rates differ between groups. Choosing a metric is therefore a value judgment about which errors are most harmful, and it should be made explicitly with domain experts and affected stakeholders rather than left to defaults.
Practical measurement starts with disaggregated evaluation: reporting accuracy, error rates and outcomes separately for each relevant group instead of one overall figure. Small group sizes produce noisy estimates, so confidence intervals should accompany each number.
AI bias compared with related safety concepts
AI bias overlaps with several other AI safety concepts but is not the same as any of them. The table clarifies how they relate.
| Concept | Main question | Link to bias |
|---|---|---|
| AI alignment | Does the system pursue intended goals | Bias is one form of misalignment |
| Explainable AI | Why did the model decide this | Explanations help locate bias sources |
| Model card | What are the model’s properties | Documents subgroup results and limits |
| AI red teaming | How can the system be made to fail | Probes for biased or harmful output |
| Statistical bias | Is an estimator systematically off | Technical sense, not always about fairness |
The word bias also has a neutral technical meaning. The bias term in a neuron and the bias-variance trade-off in statistics have nothing to do with fairness, which is a frequent source of confusion.
Mitigating AI bias in practice
Mitigation works at three points in the pipeline. No single technique removes bias, so teams usually combine several and verify the effect with disaggregated evaluation.
| Stage | Technique | Trade-off |
|---|---|---|
| Pre-processing | Rebalance, reweight or relabel data | Needs reliable group information |
| In-processing | Fairness constraints in training | May reduce overall accuracy slightly |
| Post-processing | Adjust thresholds per group | Raises legal and policy questions |
| Governance | Impact assessments, audits, appeals | Ongoing cost and expertise |
- Define harm first: decide which errors matter most before choosing metrics.
- Test on real deployment data, not only on curated benchmarks.
- Monitor after launch, because populations and behavior shift over time.
- Keep a human path: affected people need a way to contest automated decisions.
Regulations increasingly require such measures. The EU AI Act, for example, sets data governance and bias examination duties for high-risk systems, and anti-discrimination laws in many countries apply to automated decisions as they do to human ones.
A plain-language summary
An AI system learns from examples, much as a person learns from experience. If the examples are lopsided, for instance if past decisions were unfair or some groups barely appear, the system learns the same lopsided view. It then repeats that pattern every time it is used.
AI bias does not mean the computer has opinions. It means the mistakes are not evenly spread: some people are treated worse again and again. Finding it requires checking results group by group, not just looking at an overall score.
An example from practice
A city health department pilots a model that predicts which patients are likely to miss follow-up appointments, so that staff can call them in advance. Overall accuracy looks good, but a disaggregated evaluation by neighborhood shows that patients from two low-income districts are flagged far less often than their actual no-show rate would justify.
Investigation reveals that the model relied heavily on past contact history, and patients in those districts had fewer recorded phone numbers, so missing data looked like low risk. The team adds an explicit missing-data indicator, retrains with reweighted samples and chooses equal opportunity as its target metric, because the goal is to reach everyone who needs a reminder.
After the changes, true positive rates across neighborhoods fall within a few points of each other, and the department adds quarterly subgroup reviews to its monitoring routine.