A Traffic Sign Classifier — The Convolutional Neural Network

Tushar Sharma
Oct 23, 2019 · 8 min read
Image for post
Image for post
Photo by Franki Chamaki on Unsplash

Okay so as promised here’s the first part(second if you count the overview) focusing on the first thing creating a model for Image classification for the Belgium Traffic Signs Datasets.
Let’s try and break this one up into headlines. I’d try my best to explain my mindset behind the decisions.

The code we’ll be following can be found in the kaggle kernel here, and some of the experiments that led to this final kernel can be found on my github here. It does contain some experiments I did with transfer learning but they didn’t yield good results. I’m still learning and plan on improving the model further so any inputs are welcome!

Well without any further delay let's dive in!

Step 1 — Data Acquisition

Selecting the dataset is the most important part of the machine learning process so take great care as it directs the way the application goes.

It took me a while to settle on a dataset but lucky for you, you can find the dataset here. Download the image sets provided against BelgiumTS for Classification. Kaggle does have the traffic dataset incase you are working in a kernel. But, if you’re working locally or on a Colab Notebook you can extract the files into their respective folders using the following snippet:

https://gist.github.com/Tusharsharma118/bfacc092fe85540d186ea2a7a40293d5

Now, before we can begin to explore our dataset we need to load the images into memory. We do this using the imread method provided by the Scikit-learn module. We create a method that takes as input the directory path and returns two numpy arrays one for images and one for corresponding labels.
Refer the gist below for reference:

https://gist.github.com/Tusharsharma118/658d12cc22a93fe13bfd389e89f481c3

Step 2 — Exploring the Dataset

When we look at the dimensions of few images we can see that images have different dimensions over a wide range. We can use Matplotlib to visualize a few images and see their dimension and the pixel range. I defined a simple function that print a specified number of random images from the input images array.

https://gist.github.com/Tusharsharma118/6441b6dd5162acf9e8d170a6f03a14ab

Now, as the image dimensions are varying for the images in the dataset we would need to normalize the dataset to a single dimension, as expected by the model.

Dimension of Image at index 0: (74, 83, 3) 
Label for Image at index 0: 1
Number of training Images : 4575

Number of Classes : 62
Some additional tidbits about the memory requirements of data
Size of an individual image: 8

From the size of our labels set we can see that the input images have 62 unique classes. We would later need to create labels for all these 62 classes.

I tried to plot the different image dimensions and take the closest possible dimension to the highest frequency present in the dataset. In the current dataset the Mode height was 97 and width was 84 which was a little too low than expected. I decided to go forward with 128x128 as size for the input image. This size can be a hyperparameter down the line and can be experimented with.

Image for post
Image for post

Next we transform all our images to a size of 128x128 to maintain consistency.

Image for post
Image for post

Now, instead of using all our training folder images for training what I did was to combine all the images(training+test folders) and then split them into 3 sets for training, validation and test data.

Tensorflow also provides us with a Data-Generator utility that helps to train models while applying several transformations to input images. I’ve applied transformations to the input image including zooming, centre shifts (both horizontal and vertical) and shearing. We keep the fill mode to ‘nearest’ so that it fills the empty pixels with their closest available neighbours. We use the following code to preview the images generated.

Image for post
Image for post
Data Generator Output

Something to try later :- We can also try to reduce the dimensionality of our images by transforming them to grayscale instead of RGB.

4. Defining the Model

Now onto the most exciting part, here we get to try different architectures for our model and see which performs the best. The acceptance criteria I set was a validation and test set accuracy ~90% for class-wise data split.

I tried different models starting with a minimum viable model without convolutional layers. The resulting Test set accuracy was ~ 56% That gave me a minimum benchmark to incrementally improve upon.
The next step involved moving on to a Convolutional Neural Network. I tried several architectures with varying number of layers and number of activation nodes. For a recap on Convolutional Neural Networks and the maths behind them you can look at these resources here and here.

Image for post
Image for post
ConvNet Test 4
Image for post
Image for post

The losses after 30 iterations (20+5) showed higher accuracy on the training set compared to the validation set. This could represent a bias in the model.

Image for post
Image for post

After stagnating in performance near a peak value of ~88%, It was time to call upon the almighty lords of regularization to reduce the bias present in our model. Lo and behold, it was our trusty saviour Dropout that came to our rescue. You should place your dropout layer after your dense layers. On testing the current model performed best with a Dropout value of 0.5.

Image for post
Image for post

Before we begin training the model let’s talk about a little something called Callbacks! A callback allows us to interrupt the model training process depending upon the logs written by every epoch or step of the training cycle.
There are a few prebuilt easy to use callbacks provided to us by Keras including the Early Stopping callback. To use it we define a new callback while providing in values for attributes as explained below:

  • monitor : define which property in the epoch log you want the callback to
    keep an eye on.
  • patience : defines how many epochs to compute further/wait for after the minimum change has been observed in the monitored value
  • min_delta : defines the change in the monitored value after which to stop the training unless a patience value is provided. (stops after patience in that case unless a new min_delta change is detected)
  • restore_best_weights : restores the weights to the epoch where the min_delta was breached.
https://gist.github.com/Tusharsharma118/a1ce3501234f6e9a4f6dfd6d08853129

Now let’s begin the training and wait……
You should define your callbacks to stop once the validation loss starts increasing as defined in the callback above or you’ve reached accuracy close to your goals.

Image for post
Image for post
Image for post
Image for post
Model tests on test set and stratified test set

As you can see above the model accuracy has significantly increased to about ~96% on the test set and 88% on a class wise equally distributed test image set.
This indicates that a few classes are hard to predict for the model while it shows high accuracy for the other part of the input classes. The model can be further worked upon to increase Its accuracy. But, for the time being I’ve decided to freeze this model and begin the next phase of the model development. I’m looking forward to you telling me about how you’ve improved the model!

Visualising the Losses

Can’t forget to visualize the losses can we? Well to do so make sure to save the results of model training in a object(here history) and we can use that to visualize our validation and training losses. Refer the code below if you’re not following the kaggle kernel.

https://gist.github.com/Tusharsharma118/71fd206011d4a9ca9d00dddac4fd5d2f
Image for post
Image for post

Saving our Model for later

I had several methods available to save the model for future use. For now we save the model as a H5 file using the ‘save’ methods provided by the model itself.

# SAVE THE MODEL AS H5 File
model_regularized.save('final_model.h5')

The Kernel also experiments with saving using the Saved Model and Tensorflow JS libraries.

Some Insights Learned to be Implemented

Stratified Split

Later down the line I learned about inconsistency in model performance on test data. This could be due to unequal instances of class wise images, that is more images of one class makes the model perform better for that class. This in-turn makes the test cases biased towards those classes giving better performance in test set.
A possible solution to the issue would be to split all images into training, validation and test sets based on classes. That is we make sure all 62 of our classes are equally represented in all 3 splits of the dataset.

Bonus method for testing images for your model directly from the colab notebook

def predict_image(model):
uploaded = files.upload()

for file_name in uploaded.keys():
# path of the image
path = file_name

img = tf.keras.preprocessing.image.load_img(path, target_size = (128,128)) # no color mode as it defaults to rgb loads a PIL image instance

img_arr = tf.keras.preprocessing.image.img_to_array(img) # converts a PIL image to a 3D numpy array
show_img = img
#print('Input image shape :{}'.format(img_arr.shape))
img_arr = np.expand_dims(img_arr,axis=0) # change shape to match expected input
#print('Input image shape :{}'.format(img_arr.shape))

classes = model.predict(img_arr,batch_size=16)
#print( 'for Image : {}, Dimensions of Predicted Classes : {}, Classes zero index value : {}'.format(path,classes.shape,classes[0]))
plt.imshow(show_img,vmin= 0,vmax=255)
plt.axis('off')
plt.show()
predicted_class = np.argmax(classes)
print('The Image belongs to class :{}, with the description : {}'.format(predicted_class,classnames[predicted_class]))

Hey! You made it! I admit the content this time was a bit involved. But I explored and learnt a lot during these past few weeks trying to create the model. Here’s hoping that you learned something new today! Share your progress and thoughts down below.

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Sign up for Analytics Vidhya News Bytes

By Analytics Vidhya

Latest news from Analytics Vidhya on our Hackathons and some of our best articles! Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Tushar Sharma

Written by

An avid reader and voracious consumer of knowledge. Striving to be better every day. Knows some tech stuff too you know!

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Tushar Sharma

Written by

An avid reader and voracious consumer of knowledge. Striving to be better every day. Knows some tech stuff too you know!

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store