Convolutional Neuronal Network with Keras-Tuner on CIFAR 10

Published in

Analytics Vidhya

6 min readMay 6, 2020

Applying Keras-Tuner to find the best CNN structure

The Convolutional Neural Network is a supervized algorithm to analiyze and classify images data. When i started this project, I knew the basics and the structure of a CNN, but i didn’t know how many of this should I put.

The Structure of a Convolutional Neural Network

https://www.mdpi.com/1424-8220/19/22/4933/htm

Image taken from “A Convolutional Neural Network for Impact Detection and Characterization of Complex Composite Structures” by Iuliana Tabian , Hailing Fu and Zahra Sharif Khodae.

A Convolutional Neural Network consists on:

2D Conv:

https://www.pyimagesearch.com/2018/12/31/keras-conv2d-and-convolutional-layers/

The 2d takes a three-dimensional input, and make it pass on a convolution kernel, this is a matrix, wich is multiplied with the input, by this metod we can get some implicit features from the input.

A featute and at the same time a disadvantage on the Conv2D, is that we can choose the dimensionality of the output, i.e. the number of filters applied in the convolution, most of the time we choose channels 32 or 64 or even powers of 2, but this depends on the programmer.

The disadvantage occurs at the time of choosing the numbers of channels,
since in principle we don’t know the number of suitable channels for the most accurate model, and run the code for each channel is not optimal and will last to long.

Pooling(Maximum):

https://www.geeksforgeeks.org/cnn-introduction-to-pooling-layer/

The pooling layer will reduce the size of each feature map by a factor of 2, reducing the number of pixels or values in each feature map to one quarter the size.

The pooling operation is specified, rather than learned. Two common functions used in the pooling operation are:

Average Pooling: Calculate the average value for each patch on the feature map.
Maximum Pooling (or Max Pooling): Calculate the maximum value for each patch of the feature map.

The most used Pooling layer is the Max, and in this case wi’ll use this for Cifar.

Is usual to make at least two or three consecutive Convolution and Pooling steps before to pass to Flattening.

And here comes another issue, in principle we don’t know the number of convolution and pooling that we should do to get the most accurate model.

Flattening

This is probably the easiest step of all.

Basically we’re going to flatten all the feature map into a column as the image.

Fully Connected Layer

https://www.superdatascience.com/blogs/convolutional-neural-networks-cnn-step-4-full-connection

The final step of our CNN consists on take the output of the previous layer, “flatten” and turns them into a single vector that can be an output with a probablity or a class.

In this step we can choose two things:

How many nodes? Our previous layer, organized the features on a column of size n, but we can choose how many nodes will be connected on the Fully Layer and then give back the vector, usually are powers of 2. but there isn’t a magic number with the most accurate model.

How many times do we make this step? We can make as many connections as we want to get the best accuracy.

So we have certain doubts in the choice of:

The number of channels for the Convolution Kernel
The times that we should apply Conv and Pooling
The number of nodes for the Fully Connected Layer
The times that we make the Fully Connected Layer

Creating our Convolutional Neural Network

Before make our CNN we must install the corresponding packages,
For this from the Anaconda Prompt, execute this command in the command prompt or in google colab:

pip install tensorflow
______________________
pip install keras
______________________
pip install keras-tuner

Regular CNN

Import Packages

For all our code will need the next packages

Normalize Data

Before loading the data, and for a better data reading we’ll define a normalization function

Load and Preprocess Data

For this project, from keras we have the data in keras.datasets.

Regular CNN model with Keras

Initially we’re going to perform a regular CNN model with Keras. This will have 2 convolutions, both with 32 channels, and his respective poolings, the flattening, and the Fully Conected Layer with 1024 nodes.

And finally we fit our CNN with our train data, and compare it with the test data

And after a few or too many minutes depending on our computer, we’ll have the next results:

These are not such bad results, it’s a 67% of accuracy, but in my case, initially I had many doubts if I could improve its accurate by changing the parameters shown above. But this is a very long and unproductive waiting time, and in some cases our accuracy was shifting instead of going up.

KERAS-TUNER MODEL

Now we are going to make the optimal model according to keras-tuner,
and there will only be certain changes to our regular code.

All the load and preprocessing code will be the same as the regular code

Model with Keras Tuner

Here our code has a change from the previous one. This are going to be

Number of input_units on the convolution

We are making that our CNN choose the input_channel parameter, randomly, between 32 and 256, with a step of 32.

The number of Convolutions and Pooling

With this parameter will have a convnet with 2–5 convolutions

The number of nodes and connected layer on the Fully Connected Layer

So finally ours CNN model will be

RandomSearch

In the model construction, we defined the parameters that are going to be randomly choosed, but we will need some other code that will use this function now.

And our objective obviously is to get best accuracy

Fit and save the Model

Finally we fit our model

And on this way the complete code is

Results

After compile and Run our code, wi’ll have the next outputs

That is the structure choosen by our tuner to be the model with the best accuracy. And finally the result will be

Conclusion

Maybe it was just me, but in principle I wanted to find which structures were the best to achieve the best accuracy and through Keras-Tuner I can not only find a suitable structure for a Convolutional Neural Network and for any Neural Network over Keras.