Explainable Artificial Intelligence (XAI)

Explainable AI or Interpret-able AI is a technique or set of rules to understand the predictive output produced by machine learning algorithms. Machine learning models are often considered as a ‘black box’ approach which are impossible to comprehend properly. If we blindly rely on a machine learning algorithm, performance may degrade and cause a huge loss in business settings as training and test data differs. In simple problems, performance degradation might not affect much but in real life setting explainable AI can be a great savior. Explainable AI ensures that we properly understand what is happening inside the ‘black-box’ and why some AI model reaches a certain decision without keeping any blind trust on the model. It thus creates more trust and a sense of accountability.

In businesses, models which are more interpret-able like linear models are more preferred than ensemble models and neural networks as linear models are more convenient to evaluate, validate and to describe to a general people with no little or no AI knowledge. Nowadays, models are getting more advanced and complex and thus becoming more difficult to explain how they work. Some common characteristics of model interpretation are — transparency, accountability and the ease of understanding.

There are mainly 2 ways in which model interpret-ability or explain-ability can be classified —

1. Global Interpretation — It focuses on a more wider perspective to understand how the model works. It might take a eye view approach to examine the model and might not go into much deeper aspects of the model. It can find an important feature for prediction but won’t go further.
2. Local Interpretation — It emphasizes on a much specific data point and tries to find out why the predicted output behaved in a certain way due to the impact of a definite feature.

In supervised machine learning, model takes inputs a produce an output. There is a possibility that the data can be biased thus making it unreliable. As a result, we need to understand and to explain how our model is working so that we can rely on the prediction values to be accurate. There are already some existing traditional methods for explain-ability which know and implement are — data analysis, visualizations, model evaluation metrics, etc. In most of the cases, they possess some limitations and gave rise to the requirement of new libraries and methods which makes AI really explainable.

Some of the new libraries or methods are as follows —

1. LIME — A method developed by researchers to have more transparency about the things happening inside an algorithm. It is known as Local Interpret-able Model-Agnostic Explanations which finds a model that locally approximates the original model to faithfully describe the output produced by any model in an explainable way. They also introduced SP-LIME, a specific technique to give a global view of the model.
2. SHAP — It is called as Shapley Additive Explanations where the Shapley value is the mean of marginal contribution of a feature value over all possible coalitions. Coalitions are combinations of features used to approximate the shapley value of a certain feature. It is a unified approach and connects game theory with local explanations.
3. ELI5 (Explain Like I am 5) — It is a python library to visualize and debug different machine learning models using unified API. It gives an efficient way to describe black box models and has built-in support for multiple ML frameworks.
4. SKATER — A unified framework for model interpretation for all types of models to build an interpret-able machine learning system for real world use case. It is available in pypi as an open source library to unleash the hidden and learned structures of a black box model both locally and globally.
5. DeepLIFT — It differentiates each neuron through its activation functions with its reference activation function and gives contribution scores with respect to their differences. It can be found as an open source python library in pypi.

These libraries mostly uses feature importance, partial dependence plots, individual conditional expectation plots to explain less complex linear models, such as — linear regression decision trees, etc.

Written by — Salman Ibne Eunus