[CV] 12. Scale-Invariant Local Feature Extraction(2): Harris-Laplace

Published in

jun-devpBlog

4 min readDec 15, 2020

This article is the second part of the topic: scale-invariant local feature extraction. As I assume you know local feature detectors (Harris, Hessian), feature descriptors (HoG), and automatic scale selection (with LoG) by now, please visit here if you have not read the first part yet or not familiar with them before continuing to go over this article.

1. Recap

Harris and Hessian are shophisticatelly designed local feature detectors, which are used to find corners in the image. Despite they show significant performance, one huge limitation lie in them: the Harris and Hessian corner detectors are not invariant to scale.

**Figure 1. Why Harris and Hessian corner detectors are not scale-invariant, from [2]**

We also took a look at the automatic scale selection which, as its name implies, finds the most suitable scale for a point (x, y) in an image to be an interest point candidate.

Figure 2. Complete picture of auto scale selection, adapted from [1, 2]

It uses Laplacian-of-Gaussian as the signature function and detects the location of blobs with its corresponding scale, which returns the highest response, in the image.

Figure 3. Production of the list of Interest point candidates with LoG, adapted from [1, 10]

2. Harris-Laplace Detector

In order to overcome the limitation of Harris detector and make it robust to the scale changes, the Harris-Laplace detector is designed. As one can image from its name, the Harris-Laplace detectors integrates the automatic scale selection with LoG into the Harris detector. Therefore, it could detect corners regardless of the image scale, meaning that a corner in an image will be constantly detected after resizing the image.

Let’s take a look how the Harris and LoG are combined for achieving the scale invariance.

2.1 Why Harris-Laplace detector is scale-invariant?

In short, Harris-Laplace detector is a multi-scaled version of Harris detector where the Harris response is computed repeatedly on the same images but convolved with LoG filters from varying scales 𝝈. Fig 4 illustrates this process.

**FIgure 4. Visualization of Harris-Laplace detector, from [12]**

The above figure of Harris-Laplace can be summarized as two-step process.

Step 1. The Harris-Laplace detector computes interest points at multiple scales using the Harris detector. Let’s say there is a 3×3 window. Then, at each scale, a point, which outputs the largest Harris response, is chosen as interest point among all points inside the window. After computing the interest point (local maxima in the window), the window is shifted to the next point and repeat this until it goes over the whole image in each scale.

But note that, as in Harris-Laplace, the second-moment matrix is computed by convolving with Laplacian-of-Gaussian (LoG) filter, not with Gaussian filter, the second-moent matrix M and the response function R should be revised accordingly, as is shown below.

Step 2. Given the list of interest points from different scales 𝝈, Harris-Laplace detector searches interest points in the list such that they are not only the local maxima in their respective scale, but also in their neighboring scales. In other words, a interest point from step 1 is again compared with 18 points from adjacent scales (9 points from each scale) whether it is maximal or not.

After step 2, the remaining interest points are the final detection results. They are visualized as circle where the center of circle indicates the position of interest point and the radius implies the scale where it come from.

In conclusion, the Harris-Laplace is a scale-invariant detector as it searchs local features (corners) over the scale space.

The following Fig 4 is the result of three pairs of input image from Harris-Laplace. One image in a pair is rotated and scaled version of the other one. As we can observe, regardless of the shift in scale and view point, the Harris-Laplace detector locates the same local features from images.