XAI:LIME to Interpret Text Classifier NLP model

Photo by Johannes Plenio on Unsplash

XAI: Explainable AI

Artificial Intelligence made dramatic success in multiple sectors for example : Medical, Defense, Entertainment, Banking, HR and Ecommerce etc.

As a result, we are surrounding by a lot of Artificial intelligence driven applications for example: Siri, Alexa, Facebook, Flipkart, Spotify, Netflix etc.

Daily, we consume lot of AI recommendations/suggestions. For example: Listen songs on music apps recommended by AI models, buy items from E-commerce websites basis of AI model recommendations and add/follow new friends on Social media apps as per AI suggestions etc.

Further, These cognitive AI models can recognize, learn and act on their own basis of their learning from training features and hyper parameters.

But, now we are in the next age of AI, we need more accountable, auditable, reliable and interpretable AI systems. So that, we can understand the decision of the blackbox complex AI models.

Explainable AI(XAI) can help us to build next generation AI systems that are interpretable, trustworthy, reliable and easily optimizable. In XAI, we can use separate AI model to explain prediction of other AI model.

XAI has multiple techniques that can help us to interpret a blackbox AI Model for example: LIME, SHAP and Anchors.

This blog will focus only on LIME to interpret each prediction of an AI model using local data points.

LIME (Local Interpretable Model Agnostic explanations) can help us to make interpretable intelligent AI systems.

The blog will cover the following topics:

• Why Explainable AI is important
• Introduction to LIME
• Math behind LIME
• Blackbox AI Model training
• LIME to interpret AI model
• Conclusion

Why Explainable AI is important:

XAI helps us to understand the functioning of AI systems. So that, we can make reliable and accountable AI systems. It also helps to explain specific recommendations/suggestions generated by AI systems. There is a possibility of inherent bias in historical data due to caste, religious, regional or gender inequalities. XAI can help us to reduce inherent bias coming from historical data by explaining why particular predictions are given by an AI engine.

The whole world focuses on the development of Explainable, Responsible and Accountable AI systems.

In Feb 2021, “Responsible AI” future road map as part of #AIforALL program was published by NITI Ayog India, with the objective of making reliable, accountable and responsible AI systems.

In April 2017, DARPA funded the “Explainable AI (XAI) program”, aimed at improving explainability of AI decisions.

In July 2017, the Chinese government released “The Development Plan for New Generation of Artificial Intelligence” to encourage high explainability AI and strong-extensibility AI.

In May 2018, the “General Data Protection Regulation” (GDPR) was published, in which the European Union grants their citizens a “right to explanation” if they are affected by algorithmic decision-making.

XAI becomes more relevant to all stakeholders like enduser, Datascientists and AI engineers.

To release AI systems in the wild, first we need to understand why AI models give any decisions or recommendations. Also need to interpret the underlying reason of any AI model decision.

Further, AI engineers or Data-scientists can optimize the AI model as per the decision insights. This information is also important for the end user while making decisions based on AI recommendations/predictions.

For example:

• HR needs to understand what resume skill, education and project experience features are responsible for AI recommendation (selection/rejection/ranking) of a resume.
• Traffic Police need to understand which abnormal pattern is captured by the AI Computer Vision System before issue challan for any vehicle

Introduction to LIME:

LIME (Local Interpretable Model Agnostic explanations) an algorithm which can help us to interpret prediction of any blackbox AI model in a reliable way, by approximate it locally with an interpretable surrogate model “g ∈ G”, where “G” is a category of probably interpretable models, for example: linear models, decision trees etc.

Notations: Here we use “f” for blackbox complex AI model, “g” surrogate interpretable LIME model, “x ∈ Rd” be the original representation of feature which we need to explain and “ x0 ∈ {0, 1} d0” is a binary vector for its interpretable representation.

Let’s assume the “f” function of blackbox AI model is represented by boundary of blue/pink surface in the above figure. All points present in the blue region belong to one class and all the points present in the pink region belong to the second class. So, we need a complex AI model “f” to approximate this decision boundary to segregate these points at the global level. But it is hard to interpret prediction from “f” blackbox model.

Now, LIME comes into the picture. LIME models take only local points into consideration and give local interpretability, but not the global structure. In above figure the bold red cross is the query datapoint which we need to explain by using the surrogate model “g”.

LIME takes a query datapoint, also gets prediction using the blackbox “f” model, also sampling a few data points in the neighbourhood of query datapoint those also highlight by red cross and weightage them.

The points nearer to the query point are given higher weightage as compared to points far away from the query point as highlighted by red cross size.

So far we have a complex AI model “f” which is trained on complete datasets and we have another surrogate model “g” which is trained on these sample data points.

Now our goal is to fit “g” model on sampled points, such that the predictions made by the model ‘g’ on these sampled points, will approximate the predictions made by our complex AI model “f”.

As the prediction made by the surrogate model “g” on the sampled points approximates the predictions made by the model “f”, the model “g” can capture the facts on why the model “f” is giving the output prediction, for a given query point.

Math behind LIME:

Interpretable Representation of data:

In LIME, Surrogate models “g” use interpretable representation of data “x0”. So that, humans can understand the representation of data, even when blackbox AI “f” models are built on actual complex features “x”.

For example: Text data problem, text data can be converted into (0,1) interpretable representation of a binary vector “x0” by using BOW (Binary Bag of Word) vectorizer, The vectorization catches the information about the presence of words in the given text. If the word ‘wi ’ is present in the given text, its component in the vector would be 1. Otherwise, it would be 0. Even-when the blackbox AI “f” model may be built on more complex representations of data ”x” like BERT or Word2Vec word embedding or sentence embedding.

Similarly in the Image data AI problem, Image super pixels can be converted into binary vectors by using presence and absence of super pixels while the Blackbox DeepCNN model may use RGB channel images data.

Complexity Measure for Surrogate model:

Further, we use “Ω(g)” to measure complexity of interpretable “g ∈ G” model. There is a tradeoff between complexity and interpretability of “g”.

For example, “Ω(g)” in case decision trees model it will be the depth of the tree, incase of linear models, It will be represent the number of non-zero weights.

Proximity measure:

To explain why the “f” model gives any prediction for the “x” data point, surrogate model “g” can explain prediction by using local neighbour data points around “x”. So, we use πx(z) as a proximity measure between an instance z to x to select a local data point. πx(z) is similar to the exponent kernel

Local Fidelity and Interpretability:

In LIME, our goal is to minimize L(f, g, πx) while having Ω(g) be low enough to be interpretable by humans.

L(f,g,πx) → In the locality defined by ‘πx’, whatever functions ‘g’ we fit, it is not far from ‘f’.

If ‘L’ is large, it means the surrogate model ‘g’ is not approximating the model ‘f’. We have to minimize the loss function ’L’ as much as possible.

So, the goal here is to find the optimal surrogate model ’g’ that belongs to the class of easily interpretable models, such that it approximates the actual model ‘f’ closely, minimizes the loss ‘L’ and also it (ie., ‘g’) should not be a complex model

In the above function, both ‘f’ and ‘πx’ are constant. It is only ‘g’ that varies.

This is the optimization problem we have to solve. Here

G → Class of Interpretable models

L → Squared Loss (as f(x)∈R)

Πx(z) = exp{-D(x,z)2/σ2} → Exponential Kernel

The loss function is

ξ(x) = min g∈G L(f,g,πx) + Ω(g)

Ω(g) → K-Lasso → L1 regularized Linear Regression with only top ‘K’ features

Along with minimizing the loss, we also have to optimize the time complexity(i.e.. the number of non zero weights of Linear regression or depth of decision tree.)

Training of Blackbox AI Model:

In this section, we will train ML classifiers on the BBC news classification dataset. Before model training let’s discuss dataset, here we have a toy dataset shared by BBC news, It has 2225 data points and has five categories i.e sport, business, politics, tech, entertainment.

First import all dependencies and load dataset into pandas dataframe and below are the code:

Perform simple EDA to get insights from dataset and below are the code:

Train three classifiers and select the best classifiers based on best F1 score and AUC score, code here:

Now, we have selected the best classifier “Random Forest” having an F1 score: 95% and AUC score: 99%. So, Random Forest is our blackbox AI model “f”.

LIME to interpret AI model:

Import all LIME dependencies and create LIME explainer by using LimeTextExplainer with default parameters and dataset class names.

Below is the code snippet:

Lets visualize LIME interpretation and see the impact of each word (feature) in Prediction.

Unseen input for prediction: “Business news: LIC IPO to turn the insurer into India’s largest listed financial firm” and below are the code snippets:

Displaying LIME interpretation as list:

Visualizing as a plot:

See each word impact on Prediction: “firm”, “financial”, “largest” , “Business” and “news” features has highest impact on prediction as “business” class.

But if we consider domain knowledge while categorization of news article then we can see “news” and “largest” words should not be high contribution in prediction.

Lets do some experiment by removing “business” word and see its impact on each word contribution for Prediction.

After removing the “business” words from input data and seeing again the impact of remaining words on Prediction.

After removing “business” word from input data, over all probability value of “business” class changed by 0.01 from existing 0.63 to 0.62. Also impact of “news”, “India” and “insurer” has changed.

So, In this toy example you can see how LIME can help us to explain the reliability of black box AI model. LIME can help us to check reliability of training dataset. We cannot make truth-worthy model on bias dataset. Further, we can modify our dataset as per LIME findings.

Conclusion:

• LIME can prove trustworthiness or reliability of blackbox AI model
• We can also use LIME to check reliability of data set to provide trustworthy prediction
• LIME explanation can also help us to gain end user trust on our AI model

References

1. Feiyu Xu,Hans Uszkoreit,Yangzhou Du,Wei Fan Explainable AI: A Brief Survey on History, Research Areas, Approaches and Challenges
2. Dr. Matt Turek Explainable Artificial Intelligence (XAI)
3. NITI Aayog RESPONSIBLE AI #AIFORALL
4. Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin “Why Should I Trust You?” Explaining the Predictions of Any Classifier