[Paper Readthrough] Visualizing Deep Networks by Optimizing with Integrated Gradients

Photo by asoggetti on Unsplash

Overview

Original Paper: Visualizing Deep Networks by Optimizing with Integrated Gradients

Visualizing Deep Networks by Optimizing with Integrated Gradients

TL;DR

• Heatmaps are not all equals, the way they correlate to the actual processing is important
• Good correlation is fundamental to understand what the strong features, automatically learned by the CNN, are so to be able to debug the NN

The I-GOS Map is much more precise in identifying the strong features in the image, in fact

• deleting them makes the CNN accuracy drop
• deleting the pixels returned by the other heat maps but not by I-GOS make the CNN accuracy decrease very slightly

Details

Relevant Metrics

• Deletion Metrics
• Insertion Metrics

Deletion Metrics

• Consists of removing parts of the image and checking the effect on the final prediction
• The NN is expected to be able to perform the prediction correctly until some relevant region is removed, then the prediction accuracy drops

Insertion Metrics

• Consists of providing to the network only certain regions and checking the effect on the final prediction
• The NN is expected not to be able to perform the prediction correctly until some relevant region is added, then the prediction accuracy increases

Image Gradients

• Consider a given Input Image, which the NN is able to classify with high confidence
• Consider the baseline image as an Image the NN is not able to classify in any way, e.g. a black or super blurred image
• Evolving the baseline image into the input image across all the image dimensions (pixels values) produces a (huge) set of intermediate images
• Observing the NN results on these intermediate images, we expect its prediction accuracy to improve as we get closer to the actual input but not equally along all the evolution paths: in fact more evolution paths will lead to more significant improvements than other ones and that’s because they are carrying the most distinctive features
• In this context, the Gradient represents the NN accuracy change with respect to the change in the input (evolution)

Image Gradient

Strategies

1. Optimize Metrics

• Find heatmaps as a result of an optimization problem targeting the above mentioned metrics

2. Image Gradients

• I-GOS

Work in progress