An algorithmic impact assessment (AIA) is a structured, documented review of how an automated decision system could affect individuals, groups and society, carried out before deployment and repeated during operation. In AI ethics, it is the main procedural tool for turning principles such as fairness, transparency and accountability into concrete obligations, evidence and decisions.
An AIA asks who is affected, what could go wrong, how severe and likely the harm is, and which safeguards, oversight and redress mechanisms will be in place. Its output is not a single score but a record of reasoning that regulators, auditors and affected people can examine.
What an algorithmic impact assessment examines
An algorithmic impact assessment examines a system in its social and institutional context, not only its technical performance. It covers the purpose of the system, the data it uses, the decisions it supports, the people subject to those decisions and the power imbalance between the operator and those people. The idea draws on environmental, privacy and human rights impact assessments, which share the same logic of anticipating harm before acting.
The following table summarizes the core questions most AIA frameworks ask.
| Dimension | Key question | Typical evidence |
|---|---|---|
| Purpose and necessity | Is automation justified for this decision | Problem statement, alternatives considered |
| Affected populations | Who is subject to the outputs | Stakeholder map, vulnerable groups |
| Data and bias | Could the data encode historical bias | Data provenance, representation analysis |
| Fairness and accuracy | Do error rates differ across groups | Disaggregated metrics, fairness tests |
| Transparency | Can people understand and contest decisions | Notices, explanations, appeal routes |
| Human oversight | Who can override the system and when | Escalation rules, reviewer training |
| Monitoring | How will drift and incidents be detected | Metrics, thresholds, review schedule |
Ownership is a recurring design question. In practice, the deploying organization usually owns the AIA, because it controls the decision context, even when the model comes from a vendor. Procurement contracts should therefore oblige suppliers to provide training data documentation, test results and change notices, since the deployer cannot assess what it cannot see.
The assessment process step by step
An AIA follows a lifecycle process rather than a one-time checklist. It begins at the design stage, when changing the system is still cheap, and continues through procurement, deployment and retirement. Effective assessments involve affected communities and domain experts, not only the engineering team.
- Scoping: describe the system, its decision context and the level of automation.
- Risk identification: list potential harms, including allocative and representational harm.
- Risk rating: estimate severity, likelihood, reversibility and scale for each harm.
- Mitigation planning: choose technical, procedural and organizational controls.
- Consultation: gather input from affected groups, advocates and independent reviewers.
- Approval and publication: record the decision and, where required, publish a summary.
- Ongoing review: reassess after significant changes, incidents or a fixed interval.
Many frameworks assign a risk tier that sets the depth of the assessment. The table below shows a typical tiering logic; the exact levels differ by jurisdiction and organization.
| Risk tier | Example system | Typical obligations |
|---|---|---|
| Low | Internal document routing | Short self-assessment, basic logging |
| Moderate | Customer service triage | Bias testing, plain-language notice |
| High | Benefit eligibility scoring | Peer review, human review of denials, publication |
| Very high | Automated decisions on liberty or safety | External audit, ministerial or board approval |
Frameworks and legal requirements
Algorithmic impact assessments are increasingly required by law or policy, especially in the public sector. Canada’s Directive on Automated Decision-Making includes a mandatory questionnaire that assigns impact levels from I to IV. The EU AI Act requires certain deployers of high-risk systems to carry out a fundamental rights impact assessment, which complements the provider’s conformity assessment and the data protection impact assessment under the GDPR.
Other instruments shape practice without being AIAs themselves. The NIST AI Risk Management Framework offers functions for mapping, measuring and managing risk, and ISO/IEC 42001 sets requirements for an AI management system that includes impact assessment. City and state rules on automated employment decision tools add bias audit obligations in specific domains.
| Instrument | Main focus | Relation to an AIA |
|---|---|---|
| Data protection impact assessment | Risks from processing personal data | Overlaps on privacy, narrower in scope |
| Fundamental rights impact assessment | Effects on rights under EU law | A legally defined form of AIA |
| AI audit | Independent verification of claims | Tests what the AIA asserts |
| Model card | Technical documentation of a model | Supplies evidence for the AIA |
| Conformity assessment | Compliance with product requirements | Provider duty, complements deployer AIA |
Weaknesses and common failure modes
The most frequent criticism is that an AIA becomes a box-ticking exercise completed after key decisions are already made. When the assessment cannot stop or change the project, it documents risk without reducing it, which critics describe as a form of ethics washing.
| Failure mode | Consequence | Remedy |
|---|---|---|
| Late timing | Findings cannot change the design | Start at problem definition |
| Self-assessment only | Blind spots and conflicts of interest | Independent review for high tiers |
| No affected voices | Harms to minorities are missed | Structured community consultation |
| Static document | Drift and new uses go unassessed | Trigger-based and scheduled reviews |
| Aggregate metrics only | Group disparities stay hidden | Disaggregate by protected attribute |
A further limitation is uncertainty about future effects. Assessors cannot foresee every use or feedback loop, so a credible AIA states its assumptions openly and defines measurable triggers for reassessment, such as a change in error rate between groups beyond an agreed threshold.
Put simply
An algorithmic impact assessment is a careful look before you leap. Before an organization lets software help decide who gets a loan, a benefit or a job interview, it writes down who could be affected. It asks what could go wrong and how it will prevent or fix those problems.
It works much like a safety inspection for a new building. The inspection does not prove the building is perfect. It does make the builders show their reasoning and their safeguards before people move in.
In practice
A city social services department plans to use a model that ranks household applications for emergency housing support by predicted need. Before procurement, the team runs an AIA and rates the system as high impact, because errors could leave vulnerable families without shelter and decisions are hard to reverse quickly.
Testing on three years of historical cases shows that the model underestimates need for single-parent households by a noticeable margin, because past records undercounted informal housing. The department retrains with corrected data, requires a caseworker to review every application ranked below the cutoff, and publishes a plain-language summary with an appeal route. A review after six months confirms that the gap between groups has narrowed and that caseworkers overrode about one ranking in twelve, which becomes the baseline for later monitoring.