Visualizing Deep Learning with t-SNE (Tutorial and Video)

Arthur Juliani
Mar 2, 2016 · Unlisted
Image for post
Image for post

Convolutional neural networks have drastically changed the kinds of tasks that can be learned by computers, especially within the field of computer vision and classification. One of the issues with these networks however is that they are often difficult to visualize and develop an intuition of what exactly they are doing. Especially when first learning how to create models, their output, while often impressive (99% accuracy, wow!) can be mysterious.

One tool developed to help allow for visualization of these neural networks is t-SNE. By performing what is essentially dimensionality reduction, similar to principal components analysis, t-SNE is able to take the high-dimensional spaces that neural networks use and convert them into lower-dimensional spaces that humans can actually make sense of. I have begun incorporating this technique into my own work, and decided to put together an iPython notebook to walk through those interested in quickly getting visual representations of their networks as well.

Like my last tutorial, I am using the MNIST dataset as the training set. This time however, I am using Google’s Tensorflow framework to more easily train and model the network, as well as get the representations we want. Google has a series of helpful tutorials for those who are new to Tensorflow. In the notebook below, I extend one of their basic tutorials to show how we can get visuals on the network's representations as it learns.

I also decided to take this one step further, and put together a small Unity3d program that would create a 3D visualization of the network learning it’s representation of the dataset. Below you can watch as the network goes from a fuzzy representation to being able to separate the test examples into discreet groups.

UPDATE: I just released a new app that will allow you to visualize your own data in 3D using t-SNE. Read about and download it here.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store