Introduction
This post would give both Tensors, with their elegant flow, and Tensorflow with its high performance due to its C++ backend and lazy evaluation due to its computational graph execution model, center stage.
To learn more of how you can flow with the tensors, keep reading to be one with the Tensorflow.
Objectives
This Jupyter notebook would walk through how to classify handwritten digits using the opensource Google Brain’s tensorflow python library.
Further, only the low level tensorflow API would be used to demonstrate how to recognize images of digits(0 through 9) with over 99% accuracy.
Install tensorflow
This notebook would use the Google tensorflow python library. Pre-installation steps include:
- installing anaconda
- installing tensorflow on your computer with anaconda:
conda install tensorflow - Optionaly install a specific version of tensorflow
conda install tensorflow=1.0.0 - Open a file named
mnist_trainer.py. In a terminal this can be achieved with:
vi mnist_trainer.py- Next import tensorflow
#! `which python`import tensorflow as tf
Setup an interactive session
Used to yield results of the built computational graph
sess = tf.InteractiveSession()Reading the mnist data
This is data from the Mixed National Institute of Standards and Technology representing digitized handwritten digits
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets(‘MNIST_data’, one_hot=True)
width = 28 # width of the image in pixels
height = 28 # height of the image in pixels
flat = width * height # number of pixels in one image
class_output = 10 # number of possible classifications for the problem
x = tf.placeholder(tf.float32, shape=[None, flat])
y_ = tf.placeholder(tf.float32, shape=[None, class_output])First convolution layer
- A convolutional layer is used to extract features of the image which are represented as tensors
- The goal is to train weights and their biases that represents the learned important features of the images
- Weights and biases are represented as variables in tensorflow
- The actual convolution, down sampling the images, is done with max_pool
- After the learning is complete for the first layer, the layer is then activated with an activation function.
- Since deep learning mimics the brain, a function that models a neuronal activation/learning is used. The current scientific consensus is that is done in a relu like manner.
x_image = tf.reshape(x, [-1,28,28,1])
W_conv1 = tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev=0.1))
b_conv1 = tf.Variable(tf.constant(0.1, shape=[32])) # need 32 biases for 32 outputs
convolve1= tf.nn.conv2d(x_image, W_conv1, strides=[1, 1, 1, 1], padding=’SAME’) + b_conv1
h_conv1 = tf.nn.relu(convolve1)
conv1 = tf.nn.max_pool(h_conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding=’SAME’)Defining Loss Function and training the model
To know how well our network is doing, we use the cross_entropy function to measure our loss between trained and actual/test(never seen by our neural network before) input data.
test accuracy 0.9916Closing session
After the computational graph yields all its values, the session is then closed
sess.close() #finish the sessionChallenges
Our experience with Tensorflow was overrall positive. Some challenges and guides to overcome them include
Having exceptions dealing with misplaced shape
ValueError: Cannot feed value of shape (100, 784) for Tensor ‘convolutional/X:0’, which has shape ‘(?, 28, 28, 1)’This challenge becomes more and more apparent the deeper the network. Having a deeper network (.e.g. GoogleNet ) with more parameters and layers for example can lead to situations where the shapes don’t match. This is exacerbated by the fact that the final result is not known until after the static computational graph is run.
Building the computational graph step by step from small one layer onwards and testing continuously helped avoid this issue.
Non-dynamic computational graph led to non-dynamic coding practices
The very speed of tensorflow comes from its lazy evaluation of a computational graph. However, when we were building the neural network and wanted to having branching logic based on various criteria(e.e.g loss, different layer performance .etc.), it was not straightforward.
Tensorflow now has an API to do control flow but it ends up not being pythonic and adds an additional layer of complexity to the neural network logic.
