Permutation Feature Importance from Scratch

Understanding the importance of permutations in the field of explainable AI

(source: author)

If you look under the hood of the most advanced XAI methods, you will find permutations. SHAP, LIME, PDPs & ICE Plots, ALEs and Friedman's H-stat all rely on them. This is why understanding permutations and their limitations is so important to the field. So, let’s start with the simplest XAI method — permutation feature importance (PFI).

To gain a deep understanding of this approach we will:

• Calculate PFI from scratch using Python.
• Explain the choices behind the method including why we permute, repeat and which metric to use.
• Discuss the limitations of permutations.

You can find the full project on GitHub.

You may also enjoy this video on the topic. And, if you want to learn more, check out my course — XAI with Python. You can get free access if you sign up to my newsletter.

Why PFI?