Image Processing — Template Matching

Ready… Set.. Match!

(Image by Author)

From our previous articles, we have discussed several image techniques that can be used to create different segmentation and conversion in the image.

This time, how about we talk about an image processing technique that can be used the same was object tracking.

Another useful processing technique as mentioned above is the so-called Template Matching.

TEMPLATE MATCHING

Template matching refers to the image processing where we find similar templates in a source image by giving a base template to compared on.

The process of template matching is done by comparing each of the pixel values of the source image one at a time to the template image. The output would be an array of similarity values when compared to the template image.

To be able to look at the similar templates found on the image, we can find the peaks in the resulting array of the template matching.

To better understand this, let us show some examples:

Let us load a sample image:

import numpy as np
from skimage.io import imshow, imread
from skimage.color import rgb2gray
import matplotlib.pyplot as plt
sample = imread('flowers2.png')
sample_g = rgb2gray(sample)
fig, ax = plt.subplots(1,2,figsize=(10,5))
ax[0].imshow(sample)
ax[1].imshow(sample_g,cmap='gray')
ax[0].set_title('Colored Image',fontsize=15)
ax[1].set_title('Grayscale Image',fontsize=15)
plt.show()

Figure 1: Sample Image (Image by Author)

For this article, we will use our old sample of small flower bouquets. We can see that we have 7 small flower bouquets and for this example, we will try to see if our algorithm can detect the other 6 flower bouquets by using one flower bouquet as the template.

To conduct Template matching, we should first identify which patch or template we will use. An example code for patch allocation is seen:

from matplotlib.patches import Rectangle
fig, ax = plt.subplots(1,2,figsize=(10,10))
ax[0].imshow(sample_g,cmap='gray')
ax[0].add_patch(Rectangle((800, 950), 200, 200, edgecolor='r', facecolor='none'));
ax[0].set_title('Patch Location',fontsize=15)
#Showing Patch
patch = sample_g[950:1150, 800:1000]
ax[1].imshow(patch,cmap='gray')
ax[1].set_title('Patch',fontsize=15)
plt.show()

Figure 2: Template/Patch Image (Image by Author)

For the trial, let us first use the white flower as the patch where we will compare the rest of the source image/sample image.

Now that we have already a patch that we have identified. We can already pass the source image and patch/template image to the template matching algorithm.

There is already a defined function called match_template from the Scikit-Image library. We can use this to simplify our process.

from skimage.feature import match_template
sample_mt = match_template(sample_g, patch)
fig, ax = plt.subplots(1,2,figsize=(10,10))
ax[0].imshow(sample_g,cmap='gray')
ax[1].imshow(sample_mt,cmap='gray')
ax[0].set_title('Grayscale',fontsize=15)
ax[1].set_title('Template Matching',fontsize=15);

Figure 3: Performing Template Matching (Image by Author)

Figure 3 shows the results of the template matching algorithm. Notice how there are bright spots in the graph? These are actually the similarity values that were derived from template matching.

Using the similarity values from the template matching results, we can plot the parts of the source image that has a high similarity from our template image.

from skimage.feature import peak_local_max
fig, ax = plt.subplots(1,2,figsize=(10,10))
ax[0].imshow(sample_g,cmap='gray')
ax[1].imshow(sample_g,cmap='gray')
patch_width, patch_height = patch.shape
for x, y in peak_local_max(sample_mt, threshold_abs=0.6):
    rect = plt.Rectangle((y, x), patch_height, patch_width, color='r', 
                         fc='none')
    ax[1].add_patch(rect)
ax[0].set_title('Grayscale',fontsize=15)
ax[1].set_title('Template Matched',fontsize=15);

Figure 4: Plotting the Peaks (Image by Author)

Figure 4 shows the templates that were detected that are quite similar to our template image. From visual truth, we can already say we are victorious in detecting and finding the white flowers using only one template image.

The result is also dependent on the threshold that you have chosen to use. The different threshold would yield different results.

Now let us try a much larger patch!

Figure 5: Patch #2 (Image by Author)

Figure 5 shows that we have chosen a match bigger patch or template to use. This time, we are using the whole flower bouquet as the template image. Let us see if we can also detect the other bouquets.

Figure 6: Template Matching Results (Image by Author)

Figure 6 shows that indeed, we were able to detect and match the template of one flower bouquet to the different flower bouquets.

Summary

For this article, we were able to show how we can leverage the use of Image Processing to aid in Object Detection and Object Recognition. Through the examples shown, we were able to successfully detect all the small flower bouquets by only using one of the flower bouquets as a template.

Template matching can be used as a pipeline in conducting object detection for machine learning models and deep learning models.

Stay Tuned for more articles!