An AI audit is a structured, evidence-based examination of an AI system, its data and the organization that operates it, carried out to determine whether the system meets defined criteria for fairness, accuracy, robustness, transparency, privacy and legal compliance. In AI ethics, it is the main mechanism for turning abstract principles into verifiable findings that someone can be held accountable for.
The practice is also called an algorithmic audit, and when it focuses narrowly on discrimination it is often called a bias audit. Unlike an internal model evaluation, an audit is defined by independence, documented criteria and a formal report, and its conclusions are meant to be relied on by regulators, customers or the public.
What an AI audit examines
An AI audit examines the whole sociotechnical system, not only the model file. Auditors look at how training data were collected, how the model was built and validated, how outputs are used in decisions and what governance controls surround the system throughout its lifecycle.
The scope is usually divided into layers, each requiring different evidence. The table below shows the common audit layers.
| Audit layer | Key questions | Typical evidence |
|---|---|---|
| Data audit | Is the data representative, lawful and documented? | Datasheets, sampling reports, consent records |
| Model audit | Does the model perform fairly and reliably? | Test results by subgroup, robustness tests |
| Process audit | Are development and deployment controlled? | Policies, approvals, change logs |
| Outcome audit | What effects does the system have on people? | Decision logs, complaint records, impact data |
| Compliance audit | Does the system meet legal obligations? | Mapping to regulations and standards |
A process audit checks whether controls exist and are followed, while a technical audit tests the system directly. Mature audits combine both, because a well-documented process can still produce a discriminatory model, and a fair model can still be misused in deployment.
How an AI audit is conducted
An AI audit follows a defined sequence similar to financial or security audits: scoping, evidence gathering, testing, evaluation against criteria and reporting. The most consequential step is choosing the criteria, because the same system can pass one fairness metric and fail another.
- Scoping: define the system, its purpose, affected groups and the decision it supports.
- Criteria selection: choose metrics, thresholds and legal or ethical benchmarks before testing.
- Access and evidence: obtain data, model access, documentation and interviews with staff.
- Testing: run disaggregated performance tests, stress tests and explanation analyses.
- Evaluation: compare results with criteria and assess the severity of each gap.
- Reporting and remediation: document findings, recommendations and a follow-up plan.
Access level strongly shapes what an audit can prove. The following table compares the main levels of access an auditor may receive.
| Access level | What the auditor sees | What can be tested | Main limitation |
|---|---|---|---|
| Black-box access | Inputs and outputs only | Outcome disparities, consistency | Causes of bias remain hidden |
| Gray-box access | Outputs plus documentation and features | Proxy variables, data lineage | Depends on documentation quality |
| White-box access | Weights, code, training data | Explanation methods, root causes | Costly and requires confidentiality |
| Outside-the-box access | Organizational context and records | Governance and accountability | Does not test the model itself |
Typical technical tests include disaggregated error rates by protected attribute, checks of demographic parity and equalized odds, and the four-fifths rule for selection rates. Explanation tools such as SHAP or counterfactual analysis help locate proxy discrimination, for example a postal code standing in for ethnicity.
Internal, external and regulatory audits
AI audits differ by who performs them and who relies on the result. Independence increases credibility but usually reduces access, which creates a persistent trade-off between independence and depth.
| Audit type | Performed by | Independence | Typical purpose |
|---|---|---|---|
| First-party audit | Internal team or internal audit function | Low to moderate | Quality assurance before release |
| Second-party audit | Contracted external firm | Moderate | Assurance for management or clients |
| Third-party audit | Researchers, journalists, civil society | High | Public accountability |
| Regulatory audit | Supervisory authority or notified body | High | Enforcement and conformity assessment |
Regulation increasingly makes audits mandatory. New York City Local Law 144 requires annual independent bias audits of automated employment decision tools, the EU AI Act requires conformity assessments for high-risk systems, and the Digital Services Act requires independent audits of very large online platforms. ISO/IEC 42001 provides a certifiable AI management system standard, and the NIST AI Risk Management Framework is widely used as a reference for criteria.
An audit should not be confused with an algorithmic impact assessment. An impact assessment is usually performed before deployment to anticipate harms, whereas an audit verifies actual behavior against criteria, often after deployment.
Limitations and common pitfalls
AI audits are only as strong as their criteria, access and follow-through. Experienced practitioners know several recurring failure modes that weaken the value of an audit report.
| Pitfall | Cause | Remedy |
|---|---|---|
| Audit washing | Narrow scope chosen to guarantee a pass | Publish scope and criteria in advance |
| Snapshot bias | Single test of a model that later changes | Continuous monitoring and periodic reaudits |
| Missing demographic data | Protected attributes not collected | Lawful inference methods or targeted data collection |
| Metric conflict | Fairness metrics are mathematically incompatible | Justify the chosen metric for the context |
| No remediation | Findings are not linked to owners | Assign accountable owners and deadlines |
A fundamental limit is the impossibility result in fairness research: when base rates differ between groups, a classifier cannot satisfy calibration and equal error rates at the same time. Auditors therefore cannot certify a system as fair in every sense; they can only state which criteria were tested and met.
Audits of generative AI add further difficulty, because outputs are open-ended. Auditors rely on red teaming, structured prompt sets and representational harm analysis, and results depend heavily on which prompts were chosen.
In everyday language
An AI audit is like a safety inspection for a decision-making machine. Just as an inspector checks a building against fire codes, an auditor checks an AI system against agreed rules: does it treat people fairly, does it make too many mistakes, and does the company know how it works?
The inspector writes down what was tested and what failed, so others can trust the result. An audit cannot prove a system is perfect, but it shows clearly where problems are and who must fix them.
Seen in the real world
A mid-sized employer in New York uses an AI tool that ranks job applicants for customer service roles. Because the tool qualifies as an automated employment decision tool, the company hires an independent auditor before the annual renewal. The auditor receives anonymized records of about 12,000 applications together with selection outcomes and self-reported demographic categories.
The audit calculates selection rates and impact ratios for each sex and race or ethnicity category. One group shows an impact ratio of 0.72, below the four-fifths benchmark. Feature analysis shows that the model rewards uninterrupted employment history, which penalizes applicants with caregiving gaps. The employer removes that feature, retrains the model and publishes the audit summary as the law requires. A follow-up test three months later shows all impact ratios above 0.85.