AISaturdayLagos: May the Tensor Flow with you…
I know, I know. You’re getting bored of my administrative details but …
woohoo!!!
AI Saturday Lagos participated in the global #pandasSprint
Last week, we switched things up a little bit, in the morning, we went through Lesson 9 of CS231n where we talked about different CNN architectures like AlexNet, GoogLenet, ResNet etc. After which we had the #teamTensorFlow guys present their awesome introduction to Tensorflow to the rest of the class.
We joined the global #pandasSprint later in the afternoon. For the #pandasSprint, we worked in groups to fix the documentation of 5 different methods in pandas.DataFrame class.
That’s it from me guys.
Below is an overview of #teamTensorFlow’s experiments with TensorFlow and MNIST dataset.
May the Tensor Flow with you… Enjoy!
The majority of this section and colaboratory is written by todun, a team member of #teamTensorFlow. You can also follow along his code here: google colab
Introduction
After the team Wakanda, I mean Team Torch Panther or was it team PyTorch razzle dazzled the AI Saturday Lagos community with its pythonic and dynamic neural networking ways, it was time for Tensors to take center stage with its elegant flow and high performance due to its C++ back-end and “lazy evaluation”.
To learn more of how you can flow with the tensors, keep reading — we’ve got you covered.
Objectives
This Jupyter notebook would walk you through how to classify handwritten digits using the opensource Google Brain’stensorflow python library.
Further, only the low level tensorflow API would be used to demonstrate how to recognize images of digits(0 through 9) with over 99% accuracy.
What is TensorFlow?
TensorFlow is a deep learning library designed by Google Brain Team within Google’s Machine Intelligence research organization.
Things in TensorFlow works like tensor. A tensor is a generalization of vectors and matrices to potentially higher dimensions. Internally, TensorFlow represents tensors as n-dimensional arrays of base datatypes.
Graph and Sessions
TensorFlow computations are represented as dataflow graphs, i.e, Nodes are the operations (ops) and Edges are the Tensor (multidimensional arrays)
A typical Tensorflow program consists of 2 phases
- construction phase: assembling a graph (model)
- execution phase: pushing data through the graph
APIs
TensorFlow has High level Apis and Low level Apis, we decided to use the Low level Api because of the following reasons:
- Time constraints
- We think it provides better exposition on how tensors flows through a computational graph
Enough of technical jargon 😃
Installing TensorFlow
We’d be using the Google TensorFlow python library. Pre-installation steps include:
- installing anaconda
- installing tensorflow on your computer with anaconda:
conda install tensorflow - Optionaly install a specific version of tensorflow
conda install tensorflow=1.0.0
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
%matplotlib inlineSetting up an interactive session
Used to yield results of the built computational graph
sess = tf.InteractiveSession()Reading the mnist data
This is data from the Mixed National Institute of Standards and Technology (MNIST) representing digitized handwritten digits.
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets(‘MNIST_data’, one_hot=True)
width = 28 # width of the image in pixels
height = 28 # height of the image in pixels
flat = width * height # number of pixels in one image
class_output = 10 # number of possible classifications for the problem
x = tf.placeholder(tf.float32, shape=[None, flat])
y_ = tf.placeholder(tf.float32, shape=[None, class_output])First convolution layer
- A convolutional layer is used to extract features of the image which are represented as tensors
- The goal is to train weights and their biases that represents the learned important features of the images
- Weights and biases are represented as variables in tensorflow
- The actual convolution, down sampling the images, is done with max_pool
- After the learning is complete for the first layer, the layer is then activated with an activation function.
- Since deep learning mimics the brain, a function that models a neuronal activation/learning is used. The current scientific consensus is that is done in a relu like manner.
x_image = tf.reshape(x, [-1,28,28,1])
W_conv1 = tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev=0.1))
b_conv1 = tf.Variable(tf.constant(0.1, shape=[32])) # need 32 biases for 32 outputs
convolve1= tf.nn.conv2d(x_image, W_conv1, strides=[1, 1, 1, 1], padding=’SAME’) + b_conv1
h_conv1 = tf.nn.relu(convolve1)
conv1 = tf.nn.max_pool(h_conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding=’SAME’)Defining Loss Function and training the model
To know how well our network is doing, we use the cross_entropy function to measure our loss between trained and actual/test(never seen by our neural network before) input data.
You can view the full code here
test accuracy 0.9916Closing session
After the computational graph yields all its values, the session is then closed
sess.close() #finish the sessionChallenges
Our experience with Tensorflow was overrall positive. Some challenges and guides to overcome them include
Having exceptions dealing with misplaced shape
ValueError: Cannot feed value of shape (100, 784) for Tensor ‘convolutional/X:0’, which has shape ‘(?, 28, 28, 1)’This challenge becomes more and more apparent the deeper the network. Having a deeper network (.e.g. GoogleNet ) with more parameters and layers for example can lead to situations where the shapes don’t match. This is exacerbated by the fact that the final result is not known until after the static computational graph is run.
Building the computational graph step by step from small one layer onwards and testing continuously helped avoid this issue.
Non-dynamic computational graph led to non-dynamic coding practices
The very speed of tensorflow comes from its “lazy evaluation” of a computational graph. However, when we were building the neural network and wanted to having branching logic based on various criteria(e.e.g loss, different layer performance .etc.), it was not straightforward.
Tensorflow now has an API to do control flow but it ends up not being pythonic and adds an additional layer of complexity to the neural network logic.
Team Members Acknowledgement
Thanks to:
1. Ejiro Onose, an engineering student at UNILAG for putting together the presentation.
2. Tella Babatunde for wrangling the MNIST dataset into a ~99.16% accurate digit classifier
3. Ibrahim Gbadegeshin for the noteworthy presentation
4. Juwe C. Raphael, Tunde Osborne for making it possible
5. Yours truly, todun, with the jupyter notebook for fielding questions and aggregating our effort
We applaud #teamTensorFlow for all the hard work they’ve put into their beautifully written article and presentation. todun, Ejiro, Tella, Ibrahim and Juwe. Thank you!.
AISaturdayLagos wouldn’t have happened without my fellow ambassador Azeez Oluwafemi, our Partners FB Dev Circle Lagos, Vesper.ng and Intel.
A big Thanks to Nurture.AI for this amazing opportunity.
Also read how AI Saturdays is Bringing the World Together with AI
See you next week 😎.
Links to Resources
https://github.com/todun/deep-frameworks-explore.git
http://bit.ly/aisaturday_tensorflow
https://en.wikipedia.org/wiki/Lazy_evaluation
Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.
Practical deep learning for coders
A friendly introduction to Convolutional Neural Networks and Image Recognition
