Immediately Understand LIME for ML Model Explanation Part 2. LIME for Image and Text Model Interpretation
Building up intuition for using LIME to interpret Image and Text models
Part 2. LIME for Image and Text Model Interpretation
Before we start exploring how to use LIME to explain Image and Text model, let’s quickly review LIME intuition introduced in Part. 1. (Please understand Part. 1 intuition for better reading experience)
LIME Intuition Review
Normally, LIME constructs a surrogate linear regression model to approximate black-box predictions on one observation and neighborhood of the observation.
So, in order to train the surrogate linear regression model, we need build a training dataset, with the training dataset, we can resolve the surrogate linear regression model. Then we can use the coefficients of the linear regression to explain the black-box prediction.
When applying LIME to tabular data, image or text, there are different methods to build the training dataset for surrogate linear regression model, and this story will focus on explaining the methods.
LIME for Image Classification Model
Compare with tabular dataset(Part. 1 has intuition), in image domain, the only difference is training dataset building, so let’s start from it.
Build Surrogate Regression Training Dataset for Image Model
Assume we have the trained model f to recognize animal from a image.
The first step is we need segment image into different areas, below example show the image segmentation.
There are 53 segment areas in above image and each segment will be treated as a feature, and each feature represents the corresponding segmented area. Different image may have different number of segments, and skimage python library can be used to implement the image segmentation.
Next, we create the training dataset Z’, Z’ has the 53 features corresponding to 53 segment areas, and for each observation in Z’, we randomly set each feature value as 0 or 1, for example one observation in Z’ could be (x1=0, x2=1, x3=1, x4=0 , , , x53=1)
To get target values for training set Z’, firstly we need mapping each Z’ instance back to a image, assume Z’ mapping to Z. The mapping rule is, Z’ feature value 1, maps to corresponding segment area of the original image, but if 0, then we set the corresponding segment area pixels as 0 in the mapped image.
Finally we can input Z(mapped image set) into the trained model f to get target values f(Z).
Above is how we get training set for resolving Surrogate Regression, the rest LIME steps are same as handling with tabular data, please refer to Part. 1 for details.
Examples of Image Classification
The example is using Pre-Trained CNN model inception_v3 to recognize image.
import keras
import skimage.io
import skimage.segmentation
import matplotlib.pyplot as plt
import numpy as np
from keras.applications import inception_v3 as inc_net
from keras.preprocessing import image
from keras.applications.imagenet_utils import decode_predictions
%matplotlib inlinedef transform_img_fn(path_list):
out = []
for img_path in path_list:
img = image.load_img(img_path, target_size=(299, 299))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = inc_net.preprocess_input(x)
out.append(x)
return np.vstack(out)images = transform_img_fn(['Border collie.jpg'])
plt.imshow(images[0] / 2 + 0.5)inet_model = inc_net.InceptionV3()
preds = inet_model.predict(images)
for x in decode_predictions(preds)[0]:
print(x)# Print Output as Below
#('n02106166', 'Border_collie', 0.8173037)
#('n02106030', 'collie', 0.07861717)
#('n02085782', 'Japanese_spaniel', 0.006248728)
#('n02086910', 'papillon', 0.005507174)
#('n02105412', 'kelpie', 0.0031061282)
Next is to create a LIME explainer and run the explanation on model inet_model and the image.
from lime import lime_image
explainer = lime_image.LimeImageExplainer()
explanation = explainer.explain_instance(
images[0].astype('double'),
inet_model.predict,
top_labels=5,
hide_color=0,
num_samples=1000)Instance explanation, num_features=5 means we want to extract the top 5 features(segmented areas) which contribute most to predict the animal (Border_collie), in other words, our mode heavily depends on the 5 segmented areas recognize the Border_collie in the image.
temp, mask = explanation.get_image_and_mask(
explanation.top_labels[0],
positive_only=True,
num_features=5,
hide_rest=True)plt.imshow(skimage.segmentation.mark_boundaries(temp / 2 + 0.5, mask))
We also can display all the segment(53 segments in example) contributions as below heatmap, the contributions actually are the feature coefficients of the surrogate linear regression.
#Select the same class explained on the figures above.
ind = explanation.top_labels[0]
#Map each explanation weight to the corresponding superpixel
dict_heatmap = dict(explanation.local_exp[ind])
heatmap = np.vectorize(dict_heatmap.get)(explanation.segments)
#Plot. The visualization makes more sense if a symmetrical colorbar is used.
plt.imshow(heatmap, cmap = 'RdBu', vmin = -heatmap.max(), vmax = heatmap.max())
plt.colorbar()
LIME for Text Classification Model
From the experience of using LIME for image model explanation, we know creating Surrogate Regression Training dataset is the key, so let’s explore how we can create the Training dataset for text.
Intuition for Building Surrogate Regression Training Dataset for Text Classification Model
In classic NLP feature engineering, we use Bag-of-Words vector or TF-IDF vector to represent a sentence, and in this way text is converted into numeric vector which can be inputted into ML classifier.
For example, assume we have a target sentence with 5 words, and its TF-IDF vector as below, each element in the vector is a word’s TF-IDF value. Note the vector length depends on the number of unique word in all sentence collection.
(0.1, 0.2, 0.3, 0.4, 0.5, 0, 0, , , 0)
Let’s create the training dataset Z’, and each instance in Z’ is like below
(0, 1, 0, 1, 0, 0, 0, , , 0) or (1, 0, 1, 0, 1, 0, 0, , , 0), for the elements corresponding to the target sentence words, we randomly set value as 0 or 1.
Next mapping Z’ to Z, it is similar to tabular and image data, if 1 then mapping to target sentence word TF-IDF value, 0 mapping to 0, for example
Z’(0, 1, 0, 1, 0, 0, 0, , , 0) map to Z(0, 0.2, 0, 0.3, 0, 0, 0, , , 0)
Z’(1, 0, 1, 0, 1, 0, 0, , , 0) map to Z(0.1, 0, 0.3, 0, 0.4, 0, 0, , , 0)
Once we get mapped dataset Z, we can get model prediction by input Z into model f(Z).
So, we get both independent(Z’) and target values(f(Z)) to train the surrogate linear regression.
Generally speaking, what we do is just randomly remove some words from target sentence to generate the training dataset.
Also regarding the instance weight, we can use the distance between Z’ and the target sentence vector to get it.
Examples of Text Classification
Below is an example from LIME tutorial. Basic usage, two class. We explain random forest classifiers.
The model is a trained RandomForestClassifier, and text feature is text TF-IDF vector, the model is to classify input sentence into atheism or christian.
import lime
import sklearn
import numpy as np
import sklearn
import sklearn.ensemble
import sklearn.metrics
from sklearn.datasets import fetch_20newsgroups
categories = ['alt.atheism', 'soc.religion.christian']
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)
class_names = ['atheism', 'christian']vectorizer = sklearn.feature_extraction.text.TfidfVectorizer(lowercase=False)
train_vectors = vectorizer.fit_transform(newsgroups_train.data)
test_vectors = vectorizer.transform(newsgroups_test.data)rf = sklearn.ensemble.RandomForestClassifier(n_estimators=500)
rf.fit(train_vectors, newsgroups_train.target)pred = rf.predict(test_vectors)
sklearn.metrics.f1_score(newsgroups_test.target, pred, average='binary')# Model Testing F1 Score 0.9198606271777003
Create LIME text explainer and explain model classification on the 10th sentence in test dataset. The text explainer will find the top 6 words which mostly drive the model to make the classification decision.
from lime import lime_text
from sklearn.pipeline import make_pipeline
from lime.lime_text import LimeTextExplainerc = make_pipeline(vectorizer, rf)
explainer = LimeTextExplainer(class_names=class_names)idx = 10
exp = explainer.explain_instance(
newsgroups_test.data[idx],
c.predict_proba,
num_features=6)print('Document id: %d' % idx)
print('Local Regression Intercept = ', exp.intercept[1])
print('Local Regression Predict = ', exp.local_pred[0])
print('Mode Predict =', c.predict_proba([newsgroups_test.data[idx]])[0,1])
print('True class: %s' % class_names[newsgroups_test.target[idx]])#Document id: 10
#Local Regression Intercept = 0.7921681682318993
#Local Regression Predict = 0.8715259570414554
#Mode Predict = 0.878
#True class: christian
In the above example, the model predict value 0.878 and local regression predict value 0.8715 are very close, good approximating.
The top 6 words and their contribution as below, the contribution is the word’s coefficient in the approximating surrogate linear regression model, so if we sum all the 6 words coefficients plus Local Regression Intercept, the result will be close to Local Regression Predict 0.8715
exp.as_list()#[('rutgers', 0.04181413557926221),
# ('athos', 0.026596442528658143),
# ('1993', 0.021911045116494124),
# ('Christ', 0.021213631348501898),
# ('thing', -0.01908916240276232),
# ('Re', -0.016375794316893184)]
LIME also provide a good graphic picture to explain the prediction as below
exp.show_in_notebook(text=True)
Summary
This story is focus on intuition to use LIME for image and text models, and key knowledge to share is how LIME build the surrogate model training dataset for image and text. Hope you enjoy the story.
REFERENCES
- Interpretable Machine Learning: https://christophm.github.io/interpretable-ml-book/shap.html
- “Why Should I Trust You?”
Explaining the Predictions of Any Classifier : https://arxiv.org/pdf/1602.04938.pdf - The Science Behind InterpretML: LIME https://www.youtube.com/watch?v=g2WtL45-PFQ&feature=emb_rel_end
- Interpretable Machine Learning Using LIME Framework — Kasia Kulma (PhD) https://www.youtube.com/watch?v=CY3t11vuuOM&t=888s
- Understanding how LIME explains predictions https://towardsdatascience.com/understanding-how-lime-explains-predictions-d404e5d1829c
- https://github.com/marcotcr/lime
- https://data4thought.com/lime-of-words.html
- https://www.analyticsvidhya.com/blog/2020/02/quick-introduction-bag-of-words-bow-tf-idf/
