A Traffic Sign Classifier — The Convolutional Neural Network
Okay so as promised here’s the first part(second if you count the overview) focusing on the first thing creating a model for Image classification for the Belgium Traffic Signs Datasets.
Let’s try and break this one up into headlines. I’d try my best to explain my mindset behind the decisions.
The code we’ll be following can be found in the kaggle kernel here, and some of the experiments that led to this final kernel can be found on my github here. It does contain some experiments I did with transfer learning but they didn’t yield good results. I’m still learning and plan on improving the model further so any inputs are welcome!
Well without any further delay let's dive in!
Step 1 — Data Acquisition
Selecting the dataset is the most important part of the machine learning process so take great care as it directs the way the application goes.
It took me a while to settle on a dataset but lucky for you, you can find the dataset here. Download the image sets provided against BelgiumTS for Classification. Kaggle does have the traffic dataset incase you are working in a kernel. But, if you’re working locally or on a Colab Notebook you can extract the files into their respective folders using the following snippet:
Now, before we can begin to explore our dataset we need to load the images into memory. We do this using the imread method provided by the Scikit-learn module. We create a method that takes as input the directory path and returns two numpy arrays one for images and one for corresponding labels.
Refer the gist below for reference:
Step 2 — Exploring the Dataset
When we look at the dimensions of few images we can see that images have different dimensions over a wide range. We can use Matplotlib to visualize a few images and see their dimension and the pixel range. I defined a simple function that print a specified number of random images from the input images array.
Now, as the image dimensions are varying for the images in the dataset we would need to normalize the dataset to a single dimension, as expected by the model.
Dimension of Image at index 0: (74, 83, 3)
Label for Image at index 0: 1
Number of training Images : 4575
Number of Classes : 62
Some additional tidbits about the memory requirements of data
Size of an individual image: 8From the size of our labels set we can see that the input images have 62 unique classes. We would later need to create labels for all these 62 classes.
I tried to plot the different image dimensions and take the closest possible dimension to the highest frequency present in the dataset. In the current dataset the Mode height was 97 and width was 84 which was a little too low than expected. I decided to go forward with 128x128 as size for the input image. This size can be a hyperparameter down the line and can be experimented with.
Next we transform all our images to a size of 128x128 to maintain consistency.
Now, instead of using all our training folder images for training what I did was to combine all the images(training+test folders) and then split them into 3 sets for training, validation and test data.
Tensorflow also provides us with a Data-Generator utility that helps to train models while applying several transformations to input images. I’ve applied transformations to the input image including zooming, centre shifts (both horizontal and vertical) and shearing. We keep the fill mode to ‘nearest’ so that it fills the empty pixels with their closest available neighbours. We use the following code to preview the images generated.
Something to try later :- We can also try to reduce the dimensionality of our images by transforming them to grayscale instead of RGB.
4. Defining the Model
Now onto the most exciting part, here we get to try different architectures for our model and see which performs the best. The acceptance criteria I set was a validation and test set accuracy ~90% for class-wise data split.
I tried different models starting with a minimum viable model without convolutional layers. The resulting Test set accuracy was ~ 56% That gave me a minimum benchmark to incrementally improve upon.
The next step involved moving on to a Convolutional Neural Network. I tried several architectures with varying number of layers and number of activation nodes. For a recap on Convolutional Neural Networks and the maths behind them you can look at these resources here and here.
The losses after 30 iterations (20+5) showed higher accuracy on the training set compared to the validation set. This could represent a bias in the model.
After stagnating in performance near a peak value of ~88%, It was time to call upon the almighty lords of regularization to reduce the bias present in our model. Lo and behold, it was our trusty saviour Dropout that came to our rescue. You should place your dropout layer after your dense layers. On testing the current model performed best with a Dropout value of 0.5.
Before we begin training the model let’s talk about a little something called Callbacks! A callback allows us to interrupt the model training process depending upon the logs written by every epoch or step of the training cycle.
There are a few prebuilt easy to use callbacks provided to us by Keras including the Early Stopping callback. To use it we define a new callback while providing in values for attributes as explained below:
- monitor : define which property in the epoch log you want the callback to
keep an eye on. - patience : defines how many epochs to compute further/wait for after the minimum change has been observed in the monitored value
- min_delta : defines the change in the monitored value after which to stop the training unless a patience value is provided. (stops after patience in that case unless a new min_delta change is detected)
- restore_best_weights : restores the weights to the epoch where the min_delta was breached.
Now let’s begin the training and wait……
You should define your callbacks to stop once the validation loss starts increasing as defined in the callback above or you’ve reached accuracy close to your goals.
As you can see above the model accuracy has significantly increased to about ~96% on the test set and 88% on a class wise equally distributed test image set.
This indicates that a few classes are hard to predict for the model while it shows high accuracy for the other part of the input classes. The model can be further worked upon to increase Its accuracy. But, for the time being I’ve decided to freeze this model and begin the next phase of the model development. I’m looking forward to you telling me about how you’ve improved the model!
Visualising the Losses
Can’t forget to visualize the losses can we? Well to do so make sure to save the results of model training in a object(here history) and we can use that to visualize our validation and training losses. Refer the code below if you’re not following the kaggle kernel.
Saving our Model for later
I had several methods available to save the model for future use. For now we save the model as a H5 file using the ‘save’ methods provided by the model itself.
# SAVE THE MODEL AS H5 File
model_regularized.save('final_model.h5')The Kernel also experiments with saving using the Saved Model and Tensorflow JS libraries.
Some Insights Learned to be Implemented
Stratified Split
Later down the line I learned about inconsistency in model performance on test data. This could be due to unequal instances of class wise images, that is more images of one class makes the model perform better for that class. This in-turn makes the test cases biased towards those classes giving better performance in test set.
A possible solution to the issue would be to split all images into training, validation and test sets based on classes. That is we make sure all 62 of our classes are equally represented in all 3 splits of the dataset.
Bonus method for testing images for your model directly from the colab notebook
def predict_image(model):
uploaded = files.upload()
for file_name in uploaded.keys():
# path of the image
path = file_name
img = tf.keras.preprocessing.image.load_img(path, target_size = (128,128)) # no color mode as it defaults to rgb loads a PIL image instance
img_arr = tf.keras.preprocessing.image.img_to_array(img) # converts a PIL image to a 3D numpy array
show_img = img
#print('Input image shape :{}'.format(img_arr.shape))
img_arr = np.expand_dims(img_arr,axis=0) # change shape to match expected input
#print('Input image shape :{}'.format(img_arr.shape))
classes = model.predict(img_arr,batch_size=16)
#print( 'for Image : {}, Dimensions of Predicted Classes : {}, Classes zero index value : {}'.format(path,classes.shape,classes[0]))
plt.imshow(show_img,vmin= 0,vmax=255)
plt.axis('off')
plt.show()
predicted_class = np.argmax(classes)
print('The Image belongs to class :{}, with the description : {}'.format(predicted_class,classnames[predicted_class]))Hey! You made it! I admit the content this time was a bit involved. But I explored and learnt a lot during these past few weeks trying to create the model. Here’s hoping that you learned something new today! Share your progress and thoughts down below.
