Chapter 8 .1: Code for Convolutional neural networks(Tensorflow and Keras-Theano).

Last story we talked about convolutional neural networks, This story we will build the convoultional neural network using both Tensorflow and Keras (backed by Theano).

Let’s get started!!!!!!

First let’s take a problem and the dataset, we will take image classification as problem and use CIFAR10 image dataset. you can download here.

we have 10 classes and a total of 50000 training and 10000 testing images.

Let’s take the dataset into the code. in Keras , we can get the data by calling this function.

let’s understand the data and how it’s represented

so every image 3 X 32 X 32 (colors, width and height) and the values are in the image are pixel values (0–255). and there are 50000 images.

let’s convert the pixel values into between 0 and 1 to make the math easy.

Now let’s see how labels are represented

labels are class numbers (class 0, class 1 ,… class 9) so we need to convert into one hot vector (the vector length is 10 as we have 10 classes) so at the end we get the probabilities score for every class, we pick the maximum score.

Here class 6 is the label so 1 is stored at 6th index, remaining all 0's.

So the basic data understanding is done. now let’s pick any deep learning framework to implement the convolutional neural network.

TensorFlow

First let’s pick the architecture , I am gonna choose this architecture (you can try any other architecture)

placeholders for the input and output of the network. Placeholders are variables which we need to fill in when we are ready to compute the graph.

and weights for convolution 1 ,2,3 and Fully connected are initialized with the random numbers.

Let’s write the code to build the model

def model(X, W_C1, W_C2, W_C3, W_FC, W_O,p_keep_conv,p_keep_hidden):
    
    C1 = tf.nn.relu(tf.nn.conv2d(X,W_C1,
                                strides=[1,1,1,1], padding ="SAME")) 
    
    P1 = tf.nn.max_pool(C1,ksize=[1,2,2,1],
                         strides=[1,2,2,1], padding = "SAME" ) 
    
    D1 = tf.nn.dropout(P1,p_keep_conv) # 1st dropout at conv
    
    C2 = tf.nn.relu(tf.nn.conv2d(D1,W_C2,
                                strides=[1,1,1,1], padding ="SAME"))
    
    P2 = tf.nn.max_pool(C2,ksize=[1,2,2,1],
                         strides=[1,2,2,1], padding = "SAME" ) 
    
    D2 = tf.nn.dropout(P2,p_keep_conv) # 2nd dropout at conv
    
    
    C3 = tf.nn.relu(tf.nn.conv2d(D2,W_C3,
                                strides=[1,1,1,1], padding ="SAME"))
    P3 = tf.nn.max_pool(C3,ksize=[1,2,2,1],
                         strides=[1,2,2,1], padding = "SAME" ) 
    
    P3 = tf.reshape(P3, [-1, W_FC.get_shape().as_list()[0]])   
    D3 = tf.nn.dropout(P3, p_keep_conv) # 3rd dropout at conv
    
    FC = tf.nn.relu(tf.matmul(D3,W_FC))
    FC = tf.nn.dropout(FC, p_keep_hidden) #dropout at fc
    
    output = tf.matmul(FC,W_O)
    
    return output

We defined the model according to the diagram , here we used dropout , The dropout is used to prevent overfitting of the data, in the network , some % of neurons(random) don’t get computed.

next let’s define the cost function ,optimizer and predicted function.

Y_pred = model(X, W_C1, W_C2, W_C3, W_FC, W_O,p_keep_conv,p_keep_hidden)  # calling the above model.
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = Y_pred ,labels = Y))
optimizer = tf.train.RMSPropOptimizer(0.001, 0.9).minimize(cost)
predict_op = tf.argmax(Y_pred, 1)

we used here RMSProp as optimizer, but we can try others also and learning rate is 0.001, momentum=0.9.

one more thing I forgot which is we need to reshape our data according to tensorflow,

#reshape as images according to tf 
X_train = X_train.reshape(-1,32,32,3) 
X_test = X_test.reshape(-1,32,32,3)

All the variables done.

Now we need to create a session in tensorflow to run the code.

For every 10 epochs let’s try our test data to see the accuracy.

So after a few hours of laptop heating ( if CPU ) we would get the trained model with the accuracy.

Ting!!!

We can increase the accuracy by just playing with network(trying out different numbers for neurons and filters), it took me 2 hours to train the model( as I have other models also running at the same time) so I am not gonna try other, if you wish you can try, to increase the accuracy.

CIFAR 10 can get even more than 90% accuracy, if you wanna improve the accuracy , Please follow these papers (there are many ways to improve).

Let’s try out another model Using Keras(Backed by Theano).

Keras

Okay let’s try out a simple model of 2 convolution layer , 1 pooling layer and a fully connected layer.

Note: I write only required code , Full code is on my Github.

def model():
    model=Sequential()
    model.add(Convolution2D(32,3,3,activation='relu',input_shape       (3,32,32),border_mode='same',W_constraint=maxnorm(3)))
    
    model.add(Dropout(0.2))
    model.add(Convolution2D(32,3,3,activation='relu',input_shape=(3,32,32),border_mode='same',W_constraint=maxnorm(3)))
    
    model.add(MaxPooling2D(pool_size=(2,2),strides=(2,2)))
    
    model.add(Flatten())
    model.add(Dense(512,activation='relu',W_constraint=maxnorm(3)))
    model.add(Dropout(0.5))
    model.add(Dense(10,activation='softmax'))
    
    return model

Let’s call the model and check the network architecture.

Let’s do the training by calling the fit method.

model.fit(X_train, Y_train, validation_data=(X_test, Y_test), nb_epoch=epochs, batch_size=32)
# Final evaluation of the model
scores = model.evaluate(X_test, Y_test, verbose=0)
print("Accuracy: %.2f%%" % (scores[1]*100))

Woww!!!!!!!!!!

96 % accuracy. That’s hecking awesome.

you can still try to improve it Don’t give up.

That’s it for this story. Hope you enjoyed and learned something let me know if it helps .

In the next story I will cover another interesting machine leaning concept until then See Ya!

Full code is available on my Github.