Interpretable model

Also known as: glass-box model

On this page 7

An interpretable model is a machine learning model whose internal logic is directly understandable by a human, so that a person can trace how the inputs produce a specific prediction without relying on a separate explanation tool. Such models are also called glass-box models, in contrast to the black-box model, whose behavior can only be approximated from the outside.

In AI ethics, interpretability matters because it supports accountability, contestability and auditing. When a model decides on credit, hiring or medical triage, an interpretable structure lets reviewers verify that the reasoning is lawful, sensible and free of obvious proxy discrimination, instead of trusting an approximation of what the system might be doing.

What makes a model interpretable

A model is interpretable when its complete decision procedure fits in human working memory or can be inspected piece by piece in a meaningful way. Three properties usually define this: simulatability, meaning a person can compute the output by hand; decomposability, meaning each parameter or component has an intuitive meaning; and algorithmic transparency, meaning the training procedure itself is well understood.

Interpretability is therefore a property of the model form, not of a report attached afterward. A linear model with 200 interacting features can be technically transparent yet practically unreadable, so model designers also impose constraints on size and complexity, such as a limited number of terms, shallow depth or monotonic relationships.

The table below lists the dimensions on which interpretability is usually judged.

DimensionQuestion it answersTypical constraint
SimulatabilityCan a person reproduce a prediction?A handful of rules or terms
DecomposabilityDoes each part mean something?Named, meaningful input features
SparsityHow many factors drive the output?Penalties that remove weak features
MonotonicityDoes a feature always push one way?Enforced direction of effect
Global viewIs the whole model visible at once?Fits on one page or chart
Dimensions of interpretability

Common families of interpretable models

Several model families are interpretable by construction, and each trades a different amount of flexibility for transparency. Choosing among them depends on the data type, the number of features and who must read the model.

Model familyHow it is readSuitable for
Sparse linear and logistic regressionOne weight per featureRisk scores, clinical rules
Shallow decision treesPath from root to leafTriage and eligibility checks
Rule lists and rule setsOrdered if-then statementsCompliance and policy decisions
Scoring systemsInteger points added upBedside medical scores, credit checklists
Generalized additive modelsOne shape curve per featureTabular data with nonlinear effects
Case-based modelsComparison with prototypesImage tasks needing reasons by example
Interpretable model families and their typical use

Modern variants such as explainable boosting machines, a form of generalized additive model with optional pairwise interactions, often reach accuracy close to gradient boosting on tabular data. On structured problems, the accuracy gap is frequently small or absent, which challenges the widespread assumption that transparency must cost performance.

Interpretable models versus post-hoc explanation

An interpretable model explains itself, whereas a post-hoc explanation describes a black-box model from outside using methods such as SHAP, LIME, saliency maps or counterfactual explanations. The distinction matters because a post-hoc explanation is itself a model and can be unfaithful to the real logic.

AspectInterpretable modelPost-hoc explanation
Source of explanationThe model itselfA separate approximation
FidelityExact by definitionPartial, can mislead
StabilitySame answer for same inputMay vary with sampling or settings
Model flexibilityRestricted by designAny architecture allowed
Data typesStrongest on tabular dataImages, text and audio as well
Audit effortInspect the model directlyValidate both model and explainer
Interpretable models and post-hoc explanation compared

A widely cited position in the field holds that for high-stakes decisions, organizations should prefer an interpretable model whenever one performs comparably, rather than explaining a black box after the fact. The argument is practical: an explanation that is only approximately right can hide exactly the flaw an auditor needs to find.

Why interpretability supports fairness and accountability

Interpretability gives reviewers direct evidence of how a decision was reached, which is the foundation of AI accountability. It enables several concrete practices that are difficult with opaque systems.

  • Detecting proxy variables: a reviewer can see that a postal code or a gap in employment carries large weight and ask whether it stands in for a protected attribute.
  • Giving reasons to affected people: regulations on automated decisions, such as provisions in the GDPR and the EU AI Act, expect meaningful information about the logic involved.
  • Enabling contestability: a person can challenge a specific factor, for example an incorrect income figure, and the effect of correcting it is predictable.
  • Supporting audits: an AI audit or an algorithmic impact assessment can examine every rule instead of sampling behavior.

Interpretability does not guarantee fairness. A transparent model trained on biased history still reproduces historical bias, but it makes that bias visible and correctable, which a black box does not.

Limitations and misconceptions

Interpretable models have real limits, and several popular beliefs about them are inaccurate. The table below separates common claims from what practitioners actually observe.

ClaimRealityPractical consequence
Interpretable always means less accurateOften comparable on tabular dataTest a glass-box baseline first
A linear model is always interpretableNot with hundreds of correlated featuresEnforce sparsity and clear feature names
Transparency proves causalityWeights show association, not causeAvoid causal claims from coefficients
Interpretable models suit every taskWeak on raw images, audio and textUse them where features are meaningful
Openness removes the need for oversightUsers can still misread or over-trustTrain reviewers and document limits
Common claims about interpretable models

A further risk is gaming: when people know exactly which factors count, some may manipulate them. Designers must decide which features are legitimate to influence, such as paying bills on time, and which signal manipulation.

In plain words

An interpretable model is like a recipe you can read, rather than a sealed box that hands out answers. Anyone with the recipe can see which ingredients matter and how much each one counts.

When a computer helps decide who gets a loan or a hospital bed, being able to read that recipe means people can check the reasoning, spot unfair factors and fix mistakes. A black box, by contrast, only allows guesses about why it answered as it did.

A real-world example

A regional hospital wants to predict which patients are at high risk of readmission within 30 days so that nurses can schedule follow-up calls. The data team first trains a gradient-boosted model and then builds a generalized additive model on the same 40,000 past discharges.

The two models perform almost identically, with an AUC of about 0.74 each, so the hospital chooses the interpretable model. Reviewing its shape curves, clinicians notice something odd: patients with asthma appear to be at lower risk. The cause is that such patients were routinely given more intensive care in the past, which the data recorded as better outcomes.

Because the effect was visible, the team removed the misleading pattern before deployment. A black-box model could have learned the same shortcut silently and sent high-risk patients home without follow-up.