LIME

Also known as: Local interpretable model-agnostic explanations

On this page 7

LIME, short for Local interpretable model-agnostic explanations, is a post-hoc explanation technique that explains one individual prediction of any machine learning model by fitting a simple interpretable surrogate model to the model’s behavior in the immediate neighborhood of that input. Within AI ethics, LIME is one of the most widely used tools for transparency and accountability, because it lets people see which features pushed a specific decision one way or the other.

The method was introduced in 2016 by Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin. Its two defining properties are in its name: it is local, explaining a single case rather than the whole model, and it is model-agnostic, needing only the ability to query the model for predictions.

How LIME builds a local explanation

LIME builds a local explanation by perturbing the input, observing how the black-box model’s output changes, and then fitting a weighted linear model that imitates the black box only near the original instance. The coefficients of that linear model become the explanation.

  1. Select the instance to explain and convert it into an interpretable representation, such as the presence or absence of words, superpixels or binned feature values.
  2. Generate perturbed samples by randomly switching off or resampling parts of that representation, typically several thousand samples.
  3. Query the black-box model for a prediction on each sample.
  4. Weight each sample by its proximity to the original instance, usually with an exponential kernel over a distance measure.
  5. Fit a sparse linear surrogate, often limited to the top K features, using the weighted samples.
  6. Report the surrogate’s coefficients as feature contributions to this one prediction.

Formally, LIME minimizes a loss that balances local fidelity, meaning how closely the surrogate matches the black box near the instance, against complexity, meaning how many features the explanation uses. The parameters below control that balance.

ParameterWhat it controlsEffect of a poor choice
Kernel widthHow quickly sample weight decays with distanceToo wide gives a global, misleading fit; too narrow gives noise
Number of samplesSize of the perturbed datasetToo few produces unstable explanations
Number of features (K)Sparsity of the explanationToo many hurts readability; too few hides drivers
Interpretable representationWhat counts as a featurePoor choice yields meaningless explanations
Sampling distributionHow perturbations are drawnUnrealistic samples fall off the data manifold
Key parameters of LIME and their effect

LIME for text, images and tabular data

LIME adapts to different data types by changing the interpretable representation and the perturbation strategy, while the surrogate-fitting step stays the same. This flexibility is a major reason for its popularity.

Data typeInterpretable featuresPerturbation methodTypical output
TextIndividual words or tokensRemove random wordsHighlighted words with weights
ImagesSuperpixels from segmentationGray out random superpixelsImage regions that support the class
Tabular dataOriginal or discretized featuresSample from training statisticsRanked feature contributions
How LIME handles common data types

Tabular data is the most delicate case. Because samples are drawn from per-feature statistics, LIME can create unrealistic combinations, such as a ten-year-old with a mortgage, and the black box may behave arbitrarily on such points.

LIME compared with other explanation methods

LIME is one member of a family of post-hoc explanation methods, and it differs from its closest neighbors in theoretical grounding, stability and cost. Choosing among them depends on the model type and on how the explanation will be used.

MethodScopeModel access neededMain strengthMain weakness
LIMELocalPredictions onlySimple and fast to applyUnstable and kernel-dependent
SHAPLocal and globalPredictions only, faster with model internalsConsistency guarantees from Shapley valuesExpensive for many features
Integrated gradientsLocalGradientsAxiomatic attribution for neural networksRequires a baseline input
Counterfactual explanationLocalPredictions onlyShows what would change the outcomeMany valid counterfactuals exist
Partial dependence plotGlobalPredictions onlyShows average feature effectHides interactions and individual cases
LIME and related explanation techniques

Kernel SHAP can be understood as a variant of LIME with a specific kernel and weighting that yields Shapley values. In practice, many teams now prefer SHAP for tabular models and keep LIME for quick, intuitive checks on text and image classifiers.

Using LIME in fairness and accountability work

In AI ethics practice, LIME is used to inspect individual decisions for signs of bias, to support explanations given to affected people, and to debug models before deployment. It answers the question of why the model made this particular decision for this particular person.

SituationHow LIME helpsCaution
Bias investigationReveals reliance on protected attributes or proxiesAbsence of a feature does not prove fairness
Model debuggingExposes spurious cues such as watermarks or namesCheck several instances, not one
User-facing explanationLists main reasons for a decisionMust be simplified and validated
AI audit evidenceDocuments decision drivers for sample casesCombine with global fairness metrics
Typical uses of LIME in responsible AI work

A well-known demonstration showed a classifier that separated wolves from huskies by detecting snow in the background rather than the animal itself, and LIME exposed the shortcut. Such findings are valuable precisely because aggregate accuracy can hide them.

Limitations and common mistakes with LIME

The central limitation of LIME is that its explanations are approximations whose quality is not guaranteed, so they can be unstable, misleading or even deliberately manipulated. Users should treat them as hypotheses about model behavior, not as proof.

  • Instability: running LIME twice on the same instance can produce different top features because of random sampling.
  • Kernel sensitivity: results change noticeably with the kernel width, and there is no principled default for every dataset.
  • Off-manifold samples: unrealistic perturbations test the model where it was never trained.
  • Adversarial manipulation: research has shown that a model can be built to hide discriminatory behavior from LIME by detecting perturbed inputs.
  • Linear assumption: strong feature interactions cannot be captured by a local linear surrogate.

Good practice is to report fidelity scores, repeat explanations with different random seeds, and compare LIME with at least one other method before drawing conclusions about fairness.

Stripped to the essentials

LIME helps people understand why an AI made one specific decision. It takes the case in question, changes small parts of it many times, and watches how the AI’s answer shifts.

From those reactions it builds a simple, readable summary, such as: this loan was declined mostly because of high debt and a short credit history. It is like asking what would happen if one detail were different, over and over, until a pattern appears. The summary is helpful, but it is an estimate, not the full truth.

Where it is applied

A hospital data science team reviews a model that flags patients at high risk of readmission within 30 days. Before deployment, the ethics review board asks for explanations of a sample of 200 flagged and unflagged cases.

The team runs LIME on each case with 5,000 perturbed samples and the top eight features. For most patients, the explanations point to clinically sensible drivers, such as the number of prior admissions and specific diagnoses. In a cluster of cases, however, the postal code appears as a strong contributor, suggesting the model uses neighborhood as a proxy for income or ethnicity.

The team confirms the pattern with SHAP and a group fairness metric, removes the postal code and a correlated feature, and retrains. The revised model loses less than one percentage point of accuracy and shows a smaller gap in false positive rates between neighborhoods, which the board accepts for a monitored pilot.