Interpretability of Deep Learning Models with Tensorflow 2.0

An introduction to interpretability methods to ease neural network training monitoring.

Grad CAM method on ‘deer’ ImageNet class (Original photo by Asa Rodger on Unsplash)

This article dives into the tf-explain library. It provides explanations on interpretability methods, such as Grad CAM, with Tensorflow 2.0.

Did you ever wait long hours for training that turns out unsuccessful? I have, and not just once. So I began looking into tools that could help me anticipate and debug neural networks that come out of training.

This article is a summary of what I found:

• A visual logging to check pipeline integrity
• Interpretability methods to have insights on the inside of neural networks

All the methods presented in this article are implemented in tf-explain, a library built for interpretability with TensorFlow 2.0. Check out the introduction!

Introducing tf-explain, Interpretability for TensorFlow 2.0

Let’s look now at those implementations.

Look at what comes in the network

The first cause for failed trainings is simply not giving the network what you want him to have. Visualizing the inputs is crucial, and for that, I use the VisualLogging library. Dropping sample images as logs at different key moments of your pipeline (loading, resizing, data augmentation) can help you catch eventual undesired effects.

A Visual Logging setup

Logs are directly dropped into an HTML page, which you can scroll and inspect.

Example of a possible Data Augmentation Pipeline

Monitor Convolutional Kernels

Let’s now dive into interpretability methods. Key components of convolutional nets are kernels. All that matters is what the kernel is learning, how its output contributes to the final classification.

Outputs of ResNet50’s activation_1 layer for a sample cat

Intermediate Layers Visualization

A first step is to simply visualize what comes out of the activation layers. Does the output still look relevant? Or does it look like random noise? By examining how the image transits through the network, you can validate that it focuses on the right regions.

Subgraph of VGG16 to observe activations

Extracting the output of an intermediate layer with Tensorflow is fairly easy. You start from your whole model and extract a subpart of the graph. The code below shows how to obtain the outputs of the activation_1 layer from a Resnet50 model.

Intermediate Layers Visualization

Kernel Inspection

Seeing what is coming out of a layer is great, but what if we could understand what makes a kernel activate?

…

Read the original article on Sicara’s blog here.

Read the original article on Sicara’s blog here.