An interpretable model is a machine learning model whose internal logic is directly understandable by a human, so that a person can trace how the inputs produce a specific prediction without relying on a separate explanation tool. Such models are also called glass-box models, in contrast to the black-box model, whose behavior can only be approximated from the outside.
In AI ethics, interpretability matters because it supports accountability, contestability and auditing. When a model decides on credit, hiring or medical triage, an interpretable structure lets reviewers verify that the reasoning is lawful, sensible and free of obvious proxy discrimination, instead of trusting an approximation of what the system might be doing.
What makes a model interpretable
A model is interpretable when its complete decision procedure fits in human working memory or can be inspected piece by piece in a meaningful way. Three properties usually define this: simulatability, meaning a person can compute the output by hand; decomposability, meaning each parameter or component has an intuitive meaning; and algorithmic transparency, meaning the training procedure itself is well understood.
Interpretability is therefore a property of the model form, not of a report attached afterward. A linear model with 200 interacting features can be technically transparent yet practically unreadable, so model designers also impose constraints on size and complexity, such as a limited number of terms, shallow depth or monotonic relationships.
The table below lists the dimensions on which interpretability is usually judged.
| Dimension | Question it answers | Typical constraint |
|---|---|---|
| Simulatability | Can a person reproduce a prediction? | A handful of rules or terms |
| Decomposability | Does each part mean something? | Named, meaningful input features |
| Sparsity | How many factors drive the output? | Penalties that remove weak features |
| Monotonicity | Does a feature always push one way? | Enforced direction of effect |
| Global view | Is the whole model visible at once? | Fits on one page or chart |
Common families of interpretable models
Several model families are interpretable by construction, and each trades a different amount of flexibility for transparency. Choosing among them depends on the data type, the number of features and who must read the model.
| Model family | How it is read | Suitable for |
|---|---|---|
| Sparse linear and logistic regression | One weight per feature | Risk scores, clinical rules |
| Shallow decision trees | Path from root to leaf | Triage and eligibility checks |
| Rule lists and rule sets | Ordered if-then statements | Compliance and policy decisions |
| Scoring systems | Integer points added up | Bedside medical scores, credit checklists |
| Generalized additive models | One shape curve per feature | Tabular data with nonlinear effects |
| Case-based models | Comparison with prototypes | Image tasks needing reasons by example |
Modern variants such as explainable boosting machines, a form of generalized additive model with optional pairwise interactions, often reach accuracy close to gradient boosting on tabular data. On structured problems, the accuracy gap is frequently small or absent, which challenges the widespread assumption that transparency must cost performance.
Interpretable models versus post-hoc explanation
An interpretable model explains itself, whereas a post-hoc explanation describes a black-box model from outside using methods such as SHAP, LIME, saliency maps or counterfactual explanations. The distinction matters because a post-hoc explanation is itself a model and can be unfaithful to the real logic.
| Aspect | Interpretable model | Post-hoc explanation |
|---|---|---|
| Source of explanation | The model itself | A separate approximation |
| Fidelity | Exact by definition | Partial, can mislead |
| Stability | Same answer for same input | May vary with sampling or settings |
| Model flexibility | Restricted by design | Any architecture allowed |
| Data types | Strongest on tabular data | Images, text and audio as well |
| Audit effort | Inspect the model directly | Validate both model and explainer |
A widely cited position in the field holds that for high-stakes decisions, organizations should prefer an interpretable model whenever one performs comparably, rather than explaining a black box after the fact. The argument is practical: an explanation that is only approximately right can hide exactly the flaw an auditor needs to find.
Why interpretability supports fairness and accountability
Interpretability gives reviewers direct evidence of how a decision was reached, which is the foundation of AI accountability. It enables several concrete practices that are difficult with opaque systems.
- Detecting proxy variables: a reviewer can see that a postal code or a gap in employment carries large weight and ask whether it stands in for a protected attribute.
- Giving reasons to affected people: regulations on automated decisions, such as provisions in the GDPR and the EU AI Act, expect meaningful information about the logic involved.
- Enabling contestability: a person can challenge a specific factor, for example an incorrect income figure, and the effect of correcting it is predictable.
- Supporting audits: an AI audit or an algorithmic impact assessment can examine every rule instead of sampling behavior.
Interpretability does not guarantee fairness. A transparent model trained on biased history still reproduces historical bias, but it makes that bias visible and correctable, which a black box does not.
Limitations and misconceptions
Interpretable models have real limits, and several popular beliefs about them are inaccurate. The table below separates common claims from what practitioners actually observe.
| Claim | Reality | Practical consequence |
|---|---|---|
| Interpretable always means less accurate | Often comparable on tabular data | Test a glass-box baseline first |
| A linear model is always interpretable | Not with hundreds of correlated features | Enforce sparsity and clear feature names |
| Transparency proves causality | Weights show association, not cause | Avoid causal claims from coefficients |
| Interpretable models suit every task | Weak on raw images, audio and text | Use them where features are meaningful |
| Openness removes the need for oversight | Users can still misread or over-trust | Train reviewers and document limits |
A further risk is gaming: when people know exactly which factors count, some may manipulate them. Designers must decide which features are legitimate to influence, such as paying bills on time, and which signal manipulation.
In plain words
An interpretable model is like a recipe you can read, rather than a sealed box that hands out answers. Anyone with the recipe can see which ingredients matter and how much each one counts.
When a computer helps decide who gets a loan or a hospital bed, being able to read that recipe means people can check the reasoning, spot unfair factors and fix mistakes. A black box, by contrast, only allows guesses about why it answered as it did.
A real-world example
A regional hospital wants to predict which patients are at high risk of readmission within 30 days so that nurses can schedule follow-up calls. The data team first trains a gradient-boosted model and then builds a generalized additive model on the same 40,000 past discharges.
The two models perform almost identically, with an AUC of about 0.74 each, so the hospital chooses the interpretable model. Reviewing its shape curves, clinicians notice something odd: patients with asthma appear to be at lower risk. The cause is that such patients were routinely given more intensive care in the past, which the data recorded as better outcomes.
Because the effect was visible, the team removed the misleading pattern before deployment. A black-box model could have learned the same shortcut silently and sent high-risk patients home without follow-up.