Feature importance is a numerical score that estimates how much each input variable contributes to the predictions or the predictive accuracy of a trained model. In machine learning, it is one of the most widely used tools for model interpretation, helping practitioners see which signals a model relies on, detect data problems and communicate results to stakeholders in artificial intelligence projects.
Feature importance is not a single method but a family of techniques with different definitions of what “important” means. Some measure how much a variable improves the model’s internal fitting criterion, others measure how much performance drops when a variable is disrupted, and others attribute each individual prediction to its inputs. Choosing the right definition matters, because the methods can rank the same features very differently.
How feature importance scores are computed
Feature importance scores are computed either from the internal structure of the model or by probing the trained model from outside. The first group is called model-specific importance, while the second is model-agnostic and works with any predictor that accepts inputs and returns outputs.
In a decision tree, random forest or gradient boosting ensemble, the classic score is mean decrease in impurity (MDI). Every split reduces an impurity measure such as Gini impurity or variance, and the reduction is credited to the feature used for that split. Summed across all trees and normalized, these reductions give a gain-based importance. Linear models offer a simpler route: the magnitude of a standardized coefficient shows how strongly the prediction changes per standard deviation of the input.
Permutation importance takes the model-agnostic route. It randomly shuffles one feature column in a held-out dataset, breaking its relationship with the target, and measures the resulting loss in a chosen metric such as accuracy, AUC or mean squared error. A large drop means the model depends heavily on that feature. The shuffle is usually repeated 5 to 30 times to estimate the variability of the score.
The following table summarizes the main computation approaches and what each score actually quantifies.
| Method | What it measures | Model scope | Data used |
|---|---|---|---|
| Mean decrease in impurity | Total impurity reduction from splits | Tree-based models | Training data |
| Split count | Number of times a feature is used to split | Tree-based models | Training data |
| Standardized coefficients | Change in output per standard deviation | Linear and logistic models | Training data |
| Permutation importance | Metric loss after shuffling a feature | Any model | Validation or test data |
| Drop-column importance | Metric loss after retraining without a feature | Any model | Validation data, many retrains |
| SHAP values | Average contribution to each prediction | Any model, fast for trees | Any dataset |
Global and local feature importance
Feature importance can describe the model as a whole (global importance) or one specific prediction (local importance). Global scores answer which features matter on average across a dataset, while local scores explain why a particular customer, transaction or image received its specific output.
SHAP (SHapley Additive exPlanations) bridges both levels. It applies Shapley values from cooperative game theory, distributing the difference between a prediction and the average prediction fairly among the features. Averaging the absolute SHAP values over many rows yields a global ranking, while individual values explain single predictions. TreeSHAP computes exact values efficiently for tree ensembles, whereas KernelSHAP approximates them for arbitrary models at a much higher computational cost.
The table below contrasts the two levels of explanation and their typical consumers.
| Aspect | Global importance | Local importance |
|---|---|---|
| Question answered | Which features drive the model overall | Why this prediction has this value |
| Typical methods | MDI, permutation, mean absolute SHAP | SHAP values, LIME, counterfactuals |
| Typical audience | Data scientists, model reviewers | Case handlers, affected individuals, auditors |
| Main use | Feature selection, model debugging | Decision justification, error analysis |
How feature importance relates to feature selection
Feature importance is an input to feature selection, not a synonym for it. Feature selection is the decision to keep or remove variables, and importance scores are one of several signals that guide that decision, alongside correlation analysis, domain knowledge and regularization.
Regularized models such as lasso regression and elastic net perform selection implicitly by driving some coefficients to exactly zero, whereas ridge regression only shrinks them. Recursive feature elimination repeatedly trains a model, removes the least important features and retrains, using importance scores as its ranking criterion. Feature extraction methods such as principal component analysis follow a different logic: they build new composite variables, so importance then refers to components rather than original inputs, which reduces interpretability.
Choosing a feature importance method
The right method depends on the question being asked, the model type and the available computing budget. A quick diagnostic during experimentation calls for different tooling than a documented explanation for a regulated credit decision.
The following decision table maps common situations to a recommended approach and the reasoning behind it.
| Situation | Recommended method | Reason |
|---|---|---|
| Quick check of a tree ensemble | Gain-based importance | Free to compute after training |
| Comparing features fairly across types | Permutation importance on test data | Avoids bias toward high-cardinality features |
| Explaining individual decisions | SHAP values | Consistent, additive per-prediction attribution |
| Strongly correlated inputs | Grouped permutation or conditional methods | Plain shuffling creates unrealistic rows |
| Small dataset, few features | Drop-column importance | Retraining cost is affordable and exact |
| Linear model with scaled inputs | Standardized coefficients | Directly interpretable and signed |
In practice, experienced teams compare at least two methods. When gain-based and permutation rankings disagree sharply, the disagreement itself is a useful diagnostic signal that points to correlated features, overfitting or a leaking variable.
Limitations and common mistakes
Feature importance describes what the model uses, not how the world works. A high score shows predictive reliance, never causation, and a model can rely on a proxy variable that merely correlates with the true driver. Several technical pitfalls make the scores misleading even as a description of the model.
| Mistake | Consequence | Correction |
|---|---|---|
| Trusting impurity scores alone | Inflated scores for high-cardinality or noisy features | Confirm with permutation importance on held-out data |
| Computing permutation on training data | Overfitted features appear important | Use a validation or test set |
| Ignoring correlated features | Importance is split or hidden between twins | Group correlated features before scoring |
| Reading importance as causal effect | Wrong business or policy conclusions | Use causal inference methods for effects |
| Ranking from a single run | Unstable order for close scores | Repeat with several seeds and report intervals |
Data leakage often reveals itself through importance analysis. When one feature dominates with an implausibly large score, such as a timestamp or an identifier that encodes the outcome, the model has usually learned a shortcut that will fail in production. Importance scores also drift over time as data distributions change, so they should be recomputed after retraining and monitored alongside concept drift.
Put simply
Feature importance tells you which pieces of information a model pays attention to. Imagine a teacher grading essays who looks at spelling, length and argument quality. If you hid the argument quality and the grades changed sharply, you would know the teacher cares about it most.
Feature importance does the same with a computer model. It hides or scrambles one input at a time and watches how much the results suffer. The key point is that it shows what the model relies on, not what causes the outcome in real life.
In practice
A data science team at a regional bank builds a gradient boosting model to predict loan defaults from 40 applicant features. The built-in gain importance ranks an internal application ID field among the top three, which makes no business sense. Permutation importance on a held-out test set confirms the dependence: shuffling that field lowers AUC from 0.81 to 0.72.
Investigation shows that application IDs were assigned in batches, and one batch came from a period with unusually high defaults. The team removes the field, retrains and recomputes importance. Debt-to-income ratio, payment history and loan term now lead the ranking, test AUC settles at a realistic 0.78, and SHAP values provide per-applicant explanations for the credit review team. The outcome is a slightly less accurate but trustworthy model that no longer depends on a leaking identifier.