Overview Interpretable Machine Learning

Photo by Kevin Ku on Unsplash

Importance of Interpretability

If a machine learning model performs well, why do we not just trust the model and ignore why it made a certain decision? Because metrics like accuracy, F1, Precision are an incomplete description of most real-world tasks

In some cases, we don’t care why a decision was made, it is enough to know that the predictive performance on a test dataset was good. But in other cases, knowing “WHY” can help us learn more about the problem, the data and the reason why a model might fail.

Some models may not require explanations because they are used in a low-risk environment, meaning a mistake will not have serious consequences, (e.g. a movie recommender system) or the method has already been extensively studied and evaluated (e.g. optical character recognition)

By default, machine learning models pick up biases from the training data.

Different purposes of explainability in ML models sought by different audience profiles

How to interpret the model?

There are models that are interpretable by design, and those that can be explained by external techniques. This duality could also be regarded as the difference between interpretable models and model interpretability techniques, a more widely accepted classification is that of :

• Transparent models
• Post-hoc explainability

Transparent models — Level of transparency

• Simulatability denotes the ability of a model of being simulated or thought about strictly by a human, hence complexity takes a dominant place in this class. This being said, simple but extensive (i.e., with too large amount of rules) rule based systems fall out of this characteristic, whereas a single perceptron neural network falls within.
• Decomposability stands for the ability to explain each of the parts of a model (input, parameter and calculation). This characteristic might empower the ability to understand, interpret or explain the behavior of a model.
• Algorithmic transparency can be seen in different ways. It deals with the ability of the user to understand the process followed by the model to produce any given output from its input data.

Transparent models — Example

• Linear / Logistic Regression
• Decision Trees
• K-Nearest Neighbors
• Rule Based Learners
• General Additive Models
• Bayesian Models

Post-hoc explainability

When ML models do not meet any of the criteria imposed to declare them transparent, a separate method must be devised and applied to the model to explain its decisions. We can categorize into two types:

1. Model-agnostic techniques
   Techniques are designed for their application to ML models of any kind. And they’re may rely on model simplification, feature relevance explanation and visual explanation.

• Model simplification
  Almost all techniques that using this approach are based on rule extraction, and LIME: Local Interpretable Model-Agnostic Explanations is one of the example
• Feature relevance explanation
  It works by ranking or measuring the influence, relevance or importance each feature has in the prediction output by the model to be explained, e.g. SHAP (SHapley Additive exPlanations)
• Visual explanation
  Visual explanations are less common in the field of model-agnostic techniques for post-hoc explainability. Since the design of these methods must ensure that they can be seamlessly applied to any ML model disregarding its inner structure, creating visualizations from just inputs and outputs from an opaque model is a complex task. This is why almost all visualization methods falling in this category work along with feature rel-evance techniques, which provide the information that is eventually displayed to the end user.

2. Model-specific techniques
Techniques are designed for a specific ML model and thus, can not be directly extrapolated to any other learner.

Example:

• Tree ensembles, random forests
• Support Vector Machine (SVM)
• Multi-layer Neural Network
• Convolutional Neural Networks (CNN)
• Recurrent Neural Networks (RNN)

Trade-off between model interpretability and performance

Conclusion

With the increase in the number of implementation of machine learning in real life, nowadays we’re starting want to have better understanding when our model made certain predictions.

These conceptual remarks have served as a solid baseline for a systematic review of recent literature dealing with explainability, which has been approached from two different perspectives:

1. ML models that feature some degree of transparency, thereby interpretable to an extent by themselves
2. Post-hoc techniques devised to make ML models more interpretable.

References:

• [PAPER] Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
• [BOOK] Interpretable Machine Learning by Christoph Molnar