LIME, short for Local interpretable model-agnostic explanations, is a post-hoc explanation technique that explains one individual prediction of any machine learning model by fitting a simple interpretable surrogate model to the model’s behavior in the immediate neighborhood of that input. Within AI ethics, LIME is one of the most widely used tools for transparency and accountability, because it lets people see which features pushed a specific decision one way or the other.
The method was introduced in 2016 by Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin. Its two defining properties are in its name: it is local, explaining a single case rather than the whole model, and it is model-agnostic, needing only the ability to query the model for predictions.
How LIME builds a local explanation
LIME builds a local explanation by perturbing the input, observing how the black-box model’s output changes, and then fitting a weighted linear model that imitates the black box only near the original instance. The coefficients of that linear model become the explanation.
- Select the instance to explain and convert it into an interpretable representation, such as the presence or absence of words, superpixels or binned feature values.
- Generate perturbed samples by randomly switching off or resampling parts of that representation, typically several thousand samples.
- Query the black-box model for a prediction on each sample.
- Weight each sample by its proximity to the original instance, usually with an exponential kernel over a distance measure.
- Fit a sparse linear surrogate, often limited to the top K features, using the weighted samples.
- Report the surrogate’s coefficients as feature contributions to this one prediction.
Formally, LIME minimizes a loss that balances local fidelity, meaning how closely the surrogate matches the black box near the instance, against complexity, meaning how many features the explanation uses. The parameters below control that balance.
| Parameter | What it controls | Effect of a poor choice |
|---|---|---|
| Kernel width | How quickly sample weight decays with distance | Too wide gives a global, misleading fit; too narrow gives noise |
| Number of samples | Size of the perturbed dataset | Too few produces unstable explanations |
| Number of features (K) | Sparsity of the explanation | Too many hurts readability; too few hides drivers |
| Interpretable representation | What counts as a feature | Poor choice yields meaningless explanations |
| Sampling distribution | How perturbations are drawn | Unrealistic samples fall off the data manifold |
LIME for text, images and tabular data
LIME adapts to different data types by changing the interpretable representation and the perturbation strategy, while the surrogate-fitting step stays the same. This flexibility is a major reason for its popularity.
| Data type | Interpretable features | Perturbation method | Typical output |
|---|---|---|---|
| Text | Individual words or tokens | Remove random words | Highlighted words with weights |
| Images | Superpixels from segmentation | Gray out random superpixels | Image regions that support the class |
| Tabular data | Original or discretized features | Sample from training statistics | Ranked feature contributions |
Tabular data is the most delicate case. Because samples are drawn from per-feature statistics, LIME can create unrealistic combinations, such as a ten-year-old with a mortgage, and the black box may behave arbitrarily on such points.
LIME compared with other explanation methods
LIME is one member of a family of post-hoc explanation methods, and it differs from its closest neighbors in theoretical grounding, stability and cost. Choosing among them depends on the model type and on how the explanation will be used.
| Method | Scope | Model access needed | Main strength | Main weakness |
|---|---|---|---|---|
| LIME | Local | Predictions only | Simple and fast to apply | Unstable and kernel-dependent |
| SHAP | Local and global | Predictions only, faster with model internals | Consistency guarantees from Shapley values | Expensive for many features |
| Integrated gradients | Local | Gradients | Axiomatic attribution for neural networks | Requires a baseline input |
| Counterfactual explanation | Local | Predictions only | Shows what would change the outcome | Many valid counterfactuals exist |
| Partial dependence plot | Global | Predictions only | Shows average feature effect | Hides interactions and individual cases |
Kernel SHAP can be understood as a variant of LIME with a specific kernel and weighting that yields Shapley values. In practice, many teams now prefer SHAP for tabular models and keep LIME for quick, intuitive checks on text and image classifiers.
Using LIME in fairness and accountability work
In AI ethics practice, LIME is used to inspect individual decisions for signs of bias, to support explanations given to affected people, and to debug models before deployment. It answers the question of why the model made this particular decision for this particular person.
| Situation | How LIME helps | Caution |
|---|---|---|
| Bias investigation | Reveals reliance on protected attributes or proxies | Absence of a feature does not prove fairness |
| Model debugging | Exposes spurious cues such as watermarks or names | Check several instances, not one |
| User-facing explanation | Lists main reasons for a decision | Must be simplified and validated |
| AI audit evidence | Documents decision drivers for sample cases | Combine with global fairness metrics |
A well-known demonstration showed a classifier that separated wolves from huskies by detecting snow in the background rather than the animal itself, and LIME exposed the shortcut. Such findings are valuable precisely because aggregate accuracy can hide them.
Limitations and common mistakes with LIME
The central limitation of LIME is that its explanations are approximations whose quality is not guaranteed, so they can be unstable, misleading or even deliberately manipulated. Users should treat them as hypotheses about model behavior, not as proof.
- Instability: running LIME twice on the same instance can produce different top features because of random sampling.
- Kernel sensitivity: results change noticeably with the kernel width, and there is no principled default for every dataset.
- Off-manifold samples: unrealistic perturbations test the model where it was never trained.
- Adversarial manipulation: research has shown that a model can be built to hide discriminatory behavior from LIME by detecting perturbed inputs.
- Linear assumption: strong feature interactions cannot be captured by a local linear surrogate.
Good practice is to report fidelity scores, repeat explanations with different random seeds, and compare LIME with at least one other method before drawing conclusions about fairness.
Stripped to the essentials
LIME helps people understand why an AI made one specific decision. It takes the case in question, changes small parts of it many times, and watches how the AI’s answer shifts.
From those reactions it builds a simple, readable summary, such as: this loan was declined mostly because of high debt and a short credit history. It is like asking what would happen if one detail were different, over and over, until a pattern appears. The summary is helpful, but it is an estimate, not the full truth.
Where it is applied
A hospital data science team reviews a model that flags patients at high risk of readmission within 30 days. Before deployment, the ethics review board asks for explanations of a sample of 200 flagged and unflagged cases.
The team runs LIME on each case with 5,000 perturbed samples and the top eight features. For most patients, the explanations point to clinically sensible drivers, such as the number of prior admissions and specific diagnoses. In a cluster of cases, however, the postal code appears as a strong contributor, suggesting the model uses neighborhood as a proxy for income or ethnicity.
The team confirms the pattern with SHAP and a group fairness metric, removes the postal code and a correlated feature, and retrains. The revised model loses less than one percentage point of accuracy and shows a smaller gap in false positive rates between neighborhoods, which the board accepts for a monitored pilot.