TensorFlow

For my holiday break I decided to learn TensorFlow.

In the new year I have a few projects which could use a little machine learning and it seemed like a good project for a 5 day weekend.

Preparing the Environment

After doing some preliminary research I realized I would want the GPU version of TensorFlow which meant I need stuff. (GPU is about 3000 GFLOPS vs CPUs 100 GFLOPS = 30X potential performance)

GPU TensorFlow is limited to Ubuntu & Nvidia (cuda).

Set up took 3 days:

• Get GPU, realize I would need a new power supply
• Get Power supply
• Rewire PC with new power supply
• Install Ubuntu (No driver for my USB WiFi!)
• Use old TP-Link as Ethernet to WiFi Bridge
• Install Nvida Drivers, Cuda Drivers, CudaNN, Java, TensorFlow + about a dozen libraries it all needed. This sent me down many dead ends (e.g. rebuilding TensorFlow from source) and documentation driven missteps (docs say 6.5 actually needs 7.0)

Finally able to run Tensor flow get it to attach to my new GPU do some heavy workloads.

Side note: I love the new Hardware very low noise fans that stay off till TensorFlow start doing heavy work.

Simple Example (from TensorFlow docs)

Simple example of what TensorFlow is (Its written in the programming language python)

import tensorflow as tf        # import TensorFlow
import numpy as np             # import numpy for rand

# create an array “x_data” of random number
x_data = np.random.rand(100).astype(“float32”) 

# create an array “y_data” such that y_data = xdata * 0.1 + 0.3
y_data = x_data * 0.1 + 0.3

# these are the “inputs” & the job is to find the 0.1 and 0.3
# so given the arrays “x_data” and “y_data”

# create an variable “W” (set it to a random value)
W = tf.Variable(tf.random_uniform([1], -1.0, 1.0)) 

# create an variable “b” (set it to 0)
b = tf.Variable(tf.zeros([1])) 
# create a graph “y” = to W * x_data + b
y = W * x_data + b

# define "loss" as the mean square of the difference
# between the actual (y_data) and the guess (y)

loss = tf.reduce_mean(tf.square(y — y_data)) 

# define an optimizer a basic one is gradient decent (by Newton)
optimizer = tf.train.GradientDescentOptimizer(0.5)

# give the optimizer it job minimize the loss  
train = optimizer.minimize(loss) 

# define init the function which initializes all the variables 
# could just call sess.run(tf.initialize_all_variables())
init = tf.initialize_all_variables() 

sess = tf.Session()   # create the “session” 
sess.run(init)        # initialize all the variables

 
for step in xrange(201):        # loop 201 times  
 sess.run(train)                # run the optimizer
 if step % 20 == 0:             # every 20th run 
   print(step, sess.run(W), sess.run(b)) # print 

This is a simple example where we have it figure out variables that map input (x_data) to output (y_data).

TensorFlow built a “graph” : loss = reduce_mean <- square <- (W * x_data + b — y_data)

And then tried to minimize the error in the loss by gradient decent

The data can just as easily be imported images, tables etc.

The learning could be any one of a dozen approaches from Machine Learning. I tried successfully follow other examples that did Neural Net based Softmax Regression & Multilayer Convolutional Network

Within 2015 I have implemented by hand for various projects Gradient Decent & Linear Regression (a better but less general solution to this problem) TensorFlow is like a super calculator for doing this sort of thing and will likely become one of my go to tools in my work.

The big challenges left are:

• learning how to import data. (My Python is weak)
• Export trained models that I can port to run in pure Java or RenderScript (My preferred languages)