Explaining None Linear Models — Part II : The two sub-problems

Models — Part II : The two sub-problems

copyright : https://makeameme.org/

In Part I, we have discussed the motivation for explaining a ML mode, and why it is crucial in industry

In Part I, we have discussed the motivation for explaining a ML mode, and why it is crucial in industry

In this part, I am going to be a bit technical. I will try to answer the question of , what are we trying to explain exactly in the ML model ?

As it is well-known, we can break down the ML-Model building process into the following

1. Data-set (X consist of samples x_i)
2. Feature Generation (f_j)
3. Model-Selection (M)
4. Optimization algorithm that fits model to the data (Opt.)

By finishing these 4 main steps , we have a model M. So far, so good !

Let be a x’ new sample of the data , hence y’ = M(x’)

We have two main questions when it comes to explainability

1. Why does the Model M give the predictions Y’ = {y’} = M({x’}) for all samples in the test set X’ = {x’} . This is called “Global Interpretation” (The Easy problem)
2. Why does the Model M gives this single , very specific prediction y’ new for this single, very specific test-sample x’ ? This is called “Local Interpretation”

Let’s start with easier problem

Global Model Interpretability

This is done usually by feature-importance, i.e. how much each feature contribute, on average, to all Model predictions. Remember, that on-average for all predictions, that’s an importance distinction we will need later.

For ex., XGBoost provides a set of methods to do so based on feature weights (linear booster) , information gain (average and total) and so on. See get_score method in dmlc.xgb lib for more details

Local Model Interpretability

However, get_score takes only the “model” as an input , i.e , it can’t tell which features are activated/ important for a specific sample. Accordingly, what we need is a
get_score(model, sample) function.

In Part III, I am going to summarize the most important techniques of Non-Linear Model interpretability. I will be using these two wonderful resources as references

1. Explainable AI: A Review of Machine Learning Interpretability Methods
2. Interpretable Machine Learning A Guide for Making Black Box Models Explainable

So My plan is to summarize the general idea of all models, select the top 3 and provide some real-world (not text-book examples) to test these methods. Stay tuned !