QUANTRIUM GUIDES

Object Detection: Multi-Template Matching

Single or multiple object-detection in an image using list of templates

In this article, I will be throwing some light over the Multiple-Template-Matching (MTM) package and walk you through the steps involved in detecting multiple objects in an image using MTM package using one or more template images.

Introducing MTM Package

Multi-Template-Matching is a python package used to perform object-detection on a given image. It make use of list of templates as a reference to detect objects similar to them present in the given image. This package is capable of detecting multiple similar/different objects in the image as long as the templates for all the required objects to be identified are provided beforehand.

You can install the Multi-Template-Matching package using pip.

pip install Multi-Template-Matching

MTM uses OpenCV for performing different template matching techniques. You need to install OpenCV using pip as follows:

pip install opencv-python

Preparing List of Templates

The prerequisites for performing multi-template matching is the list of template images. You need to specify a tuple for an image which you intend to use as a template, the format of tuple is:

(TEMPLATE LABEL, RGB/gray-scale template image in array format)

Let’s suppose you have a folder which contains all the images which you want to use as template. Using the following code snippet you can prepare a list of template images by loading them from a specific folder. Here, name of the file is used as template label.

image_folder = PATH TO THE FOLDER HAVING TEMPLATE IMAGES
for filename in os.listdir(image_folder):
    template_img = cv2.imread(os.path.join(image_folder, filename))
    template_img = cv2.cvtColor(template_img, cv2.COLOR_BGR2GRAY)
    listTemplate.append((filename.split('.')[0], template_img))

Now, the template list is ready we can make use of the MTM package to perform the object detection.

Performing Multi-Template Matching

The matchTemplates function of the MTM package is responsible for performing Multi-Template Matching. A sample usage of matchTemplates function is as follows,

import MTM
from MTM import matchTemplates
hits = matchTemplates(listTemplate,
                      input_img,
                      score_threshold=0.9,
                      searchBox=(0, 0, 3000, 750),
                      method=cv2.TM_CCOEFF_NORMED,
                      maxOverlap=0.1)

The first parameter listTemplate corresponds to the list of template images that you have prepared in the previous step. The second parameter input_image is the target image on which multi-template matching is to be performed.

The score_threshold is the confidence threshold above which the detect object is considered as a match with one of the template images. Optimal value for score_threshold is 0.9 to minimise the risk of getting false positives.

Now if you know the region of interest in the target image where the search is to be performed, you can make use of the searchBox parameter in the format (x, y, width, height) where x and y refers to the coordinates of top left corner of the intended rectangular search space and width and height specifies the length and breadth to be considered for search. This would speed up the matching process, especially in case of where you have the large sized images with finite and smaller region of interest.

The method parameter specifies which of the OpenCV method is carried out for template matching. Following are the supported template matching techniques:

1. TM_CCOEFF → Correlation Coefficient
2. TM_CCOEFF_NORMED → Correlation Coefficient Normed
3. TM_CCORR → Cross Correlation
4. TM_CCORR_NORMED → Cross Correlation Normed
5. TM_SQDIFF_NORMED → Square Difference Normed

Note: TM_SQDIFF is not supported

The maxOverlap parameter defines the maximum ratio of intersection allowed between two detected bounding boxes. If ratio of intersection exceeds the given threshold of ratio, the lower confidence bounding box will be ignored.

Visualising the Detected Objects

The MTM package provides drawBoxesOnRGB function to draw the bounding boxes over all detected objects. You can use matplotlib library to visualise the bounding boxes. A sample usage of drawBoxesOnRGB function is as follows,

import matplotlib.pyplot as plt
from MTM import drawBoxesOnRGB
overlay = drawBoxesOnRGB(input_img,
                         hits,
                         showLabel = True,
                         labelColor=(255, 0, 0),
                         boxColor = (0, 0, 255),
                         labelScale=1,
                         boxThickness = 3)
                                           
plt.imshow(overlay)
plt.show()

Here, the first two parameters input_img and hits are the input image and the list of detected objects respectively. showLabel takes boolean value and is used to display the label for each detected object in the visualisation. Colors for the label and the bounding boxes can be set using labelColor and boxColor respectively in RGB format. You can define the size of the label using labelScale parameter. The boxThickness parameter specifies the thickness of each bounding box.

Given a template image of a clover shape with template label as clover_template, MTM module was able to detect all clovers in the below given input image.

Multiple Homogeneous Objects Detection

NOTE: Since we have provided the maxOverlap as 0.1, MTM was able to accommodate slight overlap among multiple objects

In the next example, we have provided template images of heart, clover, dice and spade, where the MTM module was able to find each one of them present in the input image

Heterogeneous Object Detection

Following is the output from matchTemplates function for the above specified input image.

Found 4 hits
                 BBox     Score     TemplateName
0  (159, 157, 87, 99)  0.999889  clover_template
3    (55, 55, 74, 84)  0.999884   spade_template
2   (163, 65, 79, 76)  0.999688   heart_template
1   (57, 158, 72, 91)  0.999484    dice_template

It is necessary to ensure that none of the template images are larger in dimensions than the input image. MTM module is sensitive and would not detect an object when there are visible variations in patterns or dimensions between the template and a similar object present in the image

Hope this illustration help you to understand the working and use of MTM package for Multiple Template Matching. Generally, it is used in manufacturing process as a part of quality control, it also used to navigate a mobile robot, or to detect edges in images. At Quantrium we have implemented this technique in object detection and document processing use cases.