Permutation importance measures a feature's importance by shuffling its values and observing how much model performance drops — a model-agnostic technique that works on any trained model, unlike tree-specific Mean Decrease in Impurity.
The Algorithm
| Step | What Happens |
|---|---|
| 1 | Measure the trained model's baseline performance on a held-out set |
| 2 | Randomly shuffle one feature's values, breaking its real relationship with the target |
| 3 | Re-measure performance with that feature shuffled — a large drop means the feature mattered a lot |
| 4 | Restore the feature, repeat for every other feature |
| 5 | Repeat the whole process several times (different random shuffles) and average, for a stable estimate |
Worked Example
A trained model's baseline validation accuracy: 0.85.
| Feature Shuffled | Accuracy After Shuffling | Importance (Drop) |
|---|---|---|
| income | 0.70 | 0.15 |
| credit_score | 0.75 | 0.10 |
| age | 0.83 | 0.02 |
| customer_id (should be irrelevant) | 0.85 | 0.00 |
Shuffling "income" costs the model 15 percentage points of accuracy — it's clearly a heavily relied-upon feature. Shuffling "customer_id" changes nothing, exactly as expected for an ID column that shouldn't carry real signal.
from sklearn.inspection import permutation_importance
import pandas as pd
result = permutation_importance(
model, X_val, y_val, n_repeats=10, random_state=42, scoring="accuracy"
)
importances = pd.DataFrame({
"feature": X_val.columns,
"importance_mean": result.importances_mean,
"importance_std": result.importances_std,
}).sort_values("importance_mean", ascending=False)
print(importances)The importance_std column matters too — a feature with high mean importance but also high variability across repeats is a less stable, less trustworthy signal than one with a similar mean but low variability.
Why This Fixes MDI's Bias
As covered in Random Forest Feature Importance, Mean Decrease in Impurity is biased toward high-cardinality features simply because they offer more split points. Permutation importance measures actual performance impact directly — a high-cardinality but genuinely useless feature (like a random ID) will show near-zero permutation importance, correctly, regardless of how many splits it could theoretically support.
Practical Use Cases
- Feature importance for any model type, including ones without a built-in importance measure (SVM, KNN, neural networks)
- A more trustworthy cross-check when MDI-based importance looks suspicious
Advantages
- Model-agnostic — works identically regardless of the underlying algorithm
- Directly measures impact on the metric you actually care about, not an internal proxy like impurity reduction
Limitations
- Computationally expensive — requires re-predicting on the full validation set once per feature, per repeat
- Can underestimate importance for groups of correlated features, since shuffling just one still leaves its correlated partners intact for the model to lean on
Common Mistakes
- Computing permutation importance on the training set instead of a held-out validation set — this can reflect overfitting rather than genuine feature usefulness.
- Running too few repeats (
n_repeats), producing a noisy, unreliable importance estimate.
Interview Relevance
Q: "Why is permutation importance considered more trustworthy than a Random Forest's built-in feature_importances_?" It directly measures the actual performance drop from removing a feature's real signal (via shuffling), on held-out data and using whatever metric you actually care about — built-in MDI importance is instead a training-time proxy (impurity reduction) that's known to be biased toward high-cardinality features.
Practice Question
A feature shows near-zero permutation importance despite being theoretically relevant to the problem. What are two possible explanations, beyond "the feature truly doesn't matter"?
