Predicting using a Caffe Classification Model with OpenCV

Starting from version 3.3, OpenCV supports the Caffe, TensorFlow, and Torch/PyTorch frameworks. Unfortunately no support to Keras model is available till this date.

Typically a Caffe model is encoded into 3 files

• A prototxt file: A text Caffe JSON file which include the model definition (deploy) (i.e. layers, expected input, …..)
• The pre-trained Caffe model : Neural Network weights.
• The labels : Text file including the labels the model is able to classify

Step#1 Load the labels

If you look at the label file, it would look as something like this:-

Sample of a Caffe label files

The first thing to do is to list label by rows as the following

Labels listed row by row

Then the classes are extracted by taking the string in each row by splitting a a space “ “ and taking the first label before the comma in cases more than one label have the same i.d. The results will be something like this

Extracted class labels

Step 2: Load the Model

net = cv2.dnn.readNetFromCaffe(prototxtFilePath,modelFilePath)

Step 3: Load and Preprocess Image

Each Caffe model mandate the shape of the input image, usually also image preprocessing is needed like mean subtraction to eliminate the effect of illumination changes. in atypical case the the mean values will be calculated from the training dataset images (Those are the mean RGB values across all images in the ImageNet dataset.)

meanValues=(104, 117, 123)
imgWidth-224
imgHeight=224
blob = cv2.dnn.blobFromImage(image, 1, (imgWidth, imgHeight), meanValues)

Step 4: Apply Inference

net.setInput(blob)
# This method returns a list of probabilities of all cases , it is of shape ((1, Number of classes)
preds = net.forward()

What is left is to sort theses probabilities indices in descending order and the top one will be the index of our class, you can uses this index to get the text label of the class.