Painless Explainability for NLP/Text Models with LIME and ELI5

Dr James Ravenscroft
Filament-Syfter
Published in
19 min readMar 14, 2022

--

understanding individual contributions of words is useful when working with NLP Classification Models

Introduction

Explainability of machine learning models is a hot topic right now — particularly in deep learning where models are that bit harder to reason about and understand. These models are often called ‘black boxes’ because you put something in, you get something out and you don’t really know how that outcome was achieved. The ability to explain machine learning model’s decisions in terms of the features passed in is both useful from a debugging standpoint (identifying features with weird weights) and with legislation like GDPR’s Right to an Explanation it is becoming important in a commercial setting to be able to explain why models behave a certain way.

In this post I will give a simplified overview of how LIME works (I may take some small technical liberties and manufacture some contrived examples to demonstrate some of these mechanisms and phenomena — apologies) and then I’ll give a brief explanation of how LIME can be applied to a sci-kit learn SVM-based sentiment model and then a huggingface/torch sentiment model.

Understanding LIME

Lime stands for Local, Interpretable Model-agnostic Explanations and is a technique proposed by Ribeiro et al. in 2016. The basic premise is that for a given input example (in an image classifier we’re…

--

--

Dr James Ravenscroft
Filament-Syfter

Ml and NLP Geek, CTO at Filament. Saxophonist, foodie and explorer. I was born in Bermuda and I Live in the UK