CNN Insights: Series conclusion. Part 7 of 7

Series links

Part 1 : Introduction

Part 2 : What do convolutional neural networks learn about images?

Part 3 : Introduction to our dataset and classification problem

Part 4 : Generating text to fit a CNN

Part 5 : Hiding input tokens to reveal classification focus

Part 6 : Scoring token-sequences by their relevance

Part 7 : Series conclusion

We looked at some techniques to interpret what CNNs were learning about images and adapted these techniques to work on text-based CNNs. Hopefully you can apply these techniques to interpret what your own CNNs are doing with your text-classification problems.

I conclude with some suggested follow-ups on interpreting what machine learning algorithms learn from text.

How to interpret interpretability?

Interpretation, insights and exploration of representations are all interesting topics which can give us confidence that our machine learning algorithms are not just getting good results but actually doing what we think they are doing. However, these topics are also a somewhat non-scientific endeavour: model interpretation is ill-defined as a task.

For instance, should interpretability mean that we can trust a model’s decisions? If this was the case then a statistical confidence level and high accuracy would be the same thing as interpretability. This does not seem to be an intuitive way of saying the model is interpretable.

If you would like to read more about the ideas around interpretability I recommend you start here:

Lipton, Zachary C. 2016. “The Mythos of Model Interpretability.” ArXiv:1606.03490 [Cs, Stat], June. http://arxiv.org/abs/1606.03490.

Local Interpretable Model-Agnostic Explanations (LIME)

LIME is a method for interpreting text classifiers. It is most similar to (but more sophisticated than) the occlusion window technique in that it explores how the classifier reacts to perturbed input. LIME treats the classifier as a black box so it works on any text classifier not just CNNs.

Introduction to Local Interpretable Model-Agnostic Explanations (LIME)

LIME is based on this paper:

Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. Why should I trust you? Explaining the predictions of any classifier In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. ACM.

The code is also available:

marcotcr/lime

CNN and images

Stanford’s CS231n course page is a great intro to CNN and images.

CS231n Convolutional Neural Networks for Visual Recognition

Checkout Francois Chollet’s blog on image generation with Keras

How convolutional neural networks see the world

I also recommend chapter 5.4 on visualising what CNNs learn from Chollet’s book on deep learning

Deep Learning with Python