Traffic Signs Recognition: CNN Model with Tensorflow Serving
In this article, we will do a Traffic Signs Recognition project with a Kaggle dataset of German traffic signs, which would require deep learning, and we’ll serve our model and test it out using Tensorflow Serving. We hit about 97% training accuracy here.
About the Dataset
The traffic signs dataset we’re gonna work with can be downloaded from Kaggle here. It contains images of 43 classes of traffic signs. Each class has a separate folder and a lot of training images within those folders. We have a similar testing dataset as well and have some additional images that we could use as well.
Retrieving Data Images and Labels
So, to read the images and get their data, we can either use the computer vision library cv2 (opencv-python) or we could go for Pillow. I’ve gone with cv2 here and you can load an image as:
import cv2
image = cv2.imread(path)Now, we want to load images for all the classes present in different folders, scale them to a standard size, and turn them into arrays of data and labels, which is done as below:
import glob
import numpy as np
import cv2data = []
labels = []
classes = 43
cur_path = '/path/to/your/dataset/directory/'for i in range(classes):
path = cur_path + 'Train/' + str(i) + '/*'
images = glob.glob(path)
for im in images:
try:
image = cv2.imread(im)
image = cv2.resize(image, (30,30))
data.append(image)
labels.append(i)
except:
print("Error loading image")data = np.array(data)
labels = np.array(labels)
Since the data is now loaded in the form of arrays, we can now perform our train test split:
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(data, labels, test_size=0.2, random_state=42)
We need a categorical encoding for our labels so we do one-hot encoding on our label data.
from tensorflow.keras.utils import to_categoricaly_train = to_categorical(y_train, 43)
y_test = to_categorical(y_test, 43)
Building the Model
Model Architecture
To efficiently build and train our model, we need to work with convolutional neural networks. The model architecture used is from Data Flair’s great work where they achieved 95% accuracy with this here. The difference in accuracies seems to be because of the usage of cv2 instead of pillow (have to test out why). The architecture is as below:
from tensorflow.keras.models import Sequential, load_model
from tensorflow.keras.layers import Conv2D, MaxPool2D, Dense, Flatten, Dropoutmodel = Sequential()
model.add(Conv2D(filters=32, kernel_size=(5,5), activation='relu', input_shape=X_train.shape[1:]))
model.add(Conv2D(filters=32, kernel_size=(5,5), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(rate=0.5))
model.add(Dense(43, activation='softmax'))
Brief Description of the Layers
We have a sequential model and for our image array data, we are using Conv2D layers with filters of 32. Filters in the first layer detect simple features and edges. As the architecture goes, the filters in the deeper layers detect more complex features that get us to our goal.
We are using ReLU as the activation function in the Conv2D layers. It basically is a piecewise linear function that outputs the value when it’s positive and returns zero if it’s negative. In our dense layer, we are using Softmax which turns our output to the probability of membership of each class and, combined, sums to 1.
MaxPool2D layer is the pooling layer used in the deep neural network. It finds and returns the maximum value for each feature map.
Compiling the Model
Next, we have to compile the model and for this, we need an ideal optimizer and a loss function. Here, we go with Adam optimizer and Categorical Crossentropy Loss as below:
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])Adam optimizer is an adaptive optimization algorithm that updates the learning rate based on moments.
The Categorical Crossentropy Loss function is used for multiclass problems. The mathematical formula for the loss function is as below:
Fitting the Model
Now we fit our model by providing the training and validation data, batch size, and epochs. For me, the best number of epochs is 18. Our accuracy becomes stable between 15 and 20 epochs, so we are choosing the number where we are getting the minimum training and validation losses.
Batch size is the number of samples before the model is updated. We are keeping it at 64 as it gives good results, but you can test this out further with your model. The model.fit code is as below:
history = model.fit(X_train, y_train, batch_size=64, epochs=18, validation_data=(X_test, y_test))Serving the Model with Tensorflow Serving
Saving the Model with Tensorflow Keras’s save_model
In order to serve the model with TensorFlow serving, we need the model to be saved in the required .pb format. For this we just save our model with our required configurations by using tf.keras.models.save_model as given below:
MODEL_DIR = '/save/model/directory/'
version = 1
export_path = os.path.join(MODEL_DIR, str(version))
tf.keras.models.save_model(
model,
export_path,
overwrite=True,
include_optimizer=True,
save_format=None,
signatures=None,
options=None
)You can then check the saved model by:
!saved_model_cli show --dir {export_path} --allInstall Tensorflow Model Server on Colab or Linux
If you are using Colab or Linux, then TensorFlow serving’s own guide recommends installing it as given below:
import sys
# We need sudo prefix if not on a Google Colab.
if 'google.colab' not in sys.modules:
SUDO_IF_NEEDED = 'sudo'
else:
SUDO_IF_NEEDED = ''!echo "deb http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | {SUDO_IF_NEEDED} tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | {SUDO_IF_NEEDED} apt-key add -
!{SUDO_IF_NEEDED} apt update!{SUDO_IF_NEEDED} apt-get install tensorflow-model-server
Tensorflow serving is not available on Windows. I use Windows and don’t use Colab so I went for WSL (Windows Subsystem for Linux) Ubuntu for my purpose. You can simply install TensorFlow-model-server with the above lines on WSL.
Serve the Model
Once you have installed the model server, it’s pretty easy to serve it now. You already defined the version folder of your model while saving it with save_model. So now you need to give the following command in your command line to serve the model:
tensorflow_model_server
--port=8500 --rest_api_port=8598
--model_name=traffic_signs_model
--model_base_path=/mnt/<drive_folder>/<model_directory>/You can see that we need our hosting port and a rest API port, we need to name our model and then provide the model base path.
The /mnt/ is basically used in WSL to access the Windows directories. You can mention your model directory as normally done if you are not on WSL.
Note that you do not need to mention the version since TensorFlow serving will automatically pick the version.
Model Inferencing
Name Classes
For proper inferences, we need to name the classes or what those labels represent. So we make a dictionary of classes.
#dictionary to label all traffic signs class.
classes = { 0:'Speed limit (20km/h)',
1:'Speed limit (30km/h)',
2:'Speed limit (50km/h)',
3:'Speed limit (60km/h)',
4:'Speed limit (70km/h)',
5:'Speed limit (80km/h)',
6:'End of speed limit (80km/h)',
7:'Speed limit (100km/h)',
8:'Speed limit (120km/h)',
9:'No passing',
10:'No passing veh over 3.5 tons',
11:'Right-of-way at intersection',
12:'Priority road',
13:'Yield',
14:'Stop',
15:'No vehicles',
16:'Veh > 3.5 tons prohibited',
17:'No entry',
18:'General caution',
19:'Dangerous curve left',
20:'Dangerous curve right',
21:'Double curve',
22:'Bumpy road',
23:'Slippery road',
24:'Road narrows on the right',
25:'Road work',
26:'Traffic signals',
27:'Pedestrians',
28:'Children crossing',
29:'Bicycles crossing',
30:'Beware of ice/snow',
31:'Wild animals crossing',
32:'End speed + passing limits',
33:'Turn right ahead',
34:'Turn left ahead',
35:'Ahead only',
36:'Go straight or right',
37:'Go straight or left',
38:'Keep right',
39:'Keep left',
40:'Roundabout mandatory',
41:'End of no passing',
42:'End no passing veh > 3.5 tons' }Load Test Images
Now we’ll load a small batch of test images. We’ll read the first five images from our test images directory and append them to a list that we can send for our predictions.
data = []
labels = []
cur_path = '/dataset/directory/'path = cur_path + 'Test/*'
images = glob.glob(path)#Retrieving the images and their labels
for im in images[:5]:
try:
image = cv2.imread(im)
image = cv2.resize(image, (30,30))
data.append(image)
except:
print("Error loading image")#Converting lists into numpy arrays
test_imgs = np.array(data)
Send Prediction Request to the Served Model
Now we create a JSON object with which we send our post request to the model directory. Define the headers as below, and send the test images as type list in the JSON object.
data = json.dumps({"signature_name": "serving_default",
"inputs": test_imgs.tolist()})headers = {"content-type": "application/json"}
Make the post request to the model as below and you’ll receive a response immediately from the model which you can then be interpreted.
json_response = requests.post(
'http://127.0.0.1:8598/v1/models/traffic_signs_model/versions/1:predict', data=data, headers=headers)predictions = json.loads(json_response.text)['outputs']
Verify the Predictions
Now, I’ll just verify the predictions for our five test images. We can do this by using our classes dictionary defined before and argmax on the predicted probabilities with:
for pred in predictions:
print(np.argmax(pred), classes[np.argmax(pred)])
We can visualize this better by displaying and matching their identified class as below:
for pred, img in zip(predictions, test_imgs):
plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
plt.title('Class ' + str(np.argmax(pred)) + ' : '
+ classes[np.argmax(pred)])
plt.show()This will give us our 5 images with their label IDs as titles.
Seems pretty accurate…
This is just to check inference for the first five images. We can make a post request with all the test images, save the prediction argmax for each image to a list and make use of the already labeled test image class ids given in the Test.csv file to compare the accuracy score with:
from sklearn.metrics import accuracy_scoreprint(accuracy_score(labels, predicted_classes_list))
Complete code can be found on Github.
References for More Learning
- Tensorflow Serving Guide
- Convolutional Neural Network (CNN)
- Deploying Multiple Models Via Tensorflow Serving
Previous Article
About Ahmad Sachal
- LinkedIn Profile — linkedin.com/in/ahmad-sachal/
About Red Buffer
- Website — redbuffer.net
- LinkedIn Page — linkedin.com/company/red-buffer
