jun-devpBlog
Published in

jun-devpBlog

[CV] 12. Scale-Invariant Local Feature Extraction(2): Harris-Laplace

This article is the second part of the topic: scale-invariant local feature extraction. As I assume you know local feature detectors (Harris, Hessian), feature descriptors (HoG), and automatic scale selection (with LoG) by now, please visit here if you have not read the first part yet or not familiar with them before continuing to go over this article.

1. Recap

Harris and Hessian are shophisticatelly designed local feature detectors, which are used to find corners in the image. Despite they show significant performance, one huge limitation lie in them: the Harris and Hessian corner detectors are not invariant to scale.

Figure 1. Why Harris and Hessian corner detectors are not scale-invariant, from [2]

We also took a look at the automatic scale selection which, as its name implies, finds the most suitable scale for a point (x, y) in an image to be an interest point candidate.

Figure 2. Complete picture of auto scale selection, adapted from [1, 2]

It uses Laplacian-of-Gaussian as the signature function and detects the location of blobs with its corresponding scale, which returns the highest response, in the image.

Figure 3. Production of the list of Interest point candidates with LoG, adapted from [1, 10]

2. Harris-Laplace Detector

In order to overcome the limitation of Harris detector and make it robust to the scale changes, the Harris-Laplace detector is designed. As one can image from its name, the Harris-Laplace detectors integrates the automatic scale selection with LoG into the Harris detector. Therefore, it could detect corners regardless of the image scale, meaning that a corner in an image will be constantly detected after resizing the image.

Let’s take a look how the Harris and LoG are combined for achieving the scale invariance.

2.1 Why Harris-Laplace detector is scale-invariant?

In short, Harris-Laplace detector is a multi-scaled version of Harris detector where the Harris response is computed repeatedly on the same images but convolved with LoG filters from varying scales 𝝈. Fig 4 illustrates this process.

FIgure 4. Visualization of Harris-Laplace detector, from [12]

The above figure of Harris-Laplace can be summarized as two-step process.

  • Step 1. The Harris-Laplace detector computes interest points at multiple scales using the Harris detector. Let’s say there is a 3×3 window. Then, at each scale, a point, which outputs the largest Harris response, is chosen as interest point among all points inside the window. After computing the interest point (local maxima in the window), the window is shifted to the next point and repeat this until it goes over the whole image in each scale.

But note that, as in Harris-Laplace, the second-moment matrix is computed by convolving with Laplacian-of-Gaussian (LoG) filter, not with Gaussian filter, the second-moent matrix M and the response function R should be revised accordingly, as is shown below.

  • Step 2. Given the list of interest points from different scales 𝝈, Harris-Laplace detector searches interest points in the list such that they are not only the local maxima in their respective scale, but also in their neighboring scales. In other words, a interest point from step 1 is again compared with 18 points from adjacent scales (9 points from each scale) whether it is maximal or not.

After step 2, the remaining interest points are the final detection results. They are visualized as circle where the center of circle indicates the position of interest point and the radius implies the scale where it come from.

In conclusion, the Harris-Laplace is a scale-invariant detector as it searchs local features (corners) over the scale space.

The following Fig 4 is the result of three pairs of input image from Harris-Laplace. One image in a pair is rotated and scaled version of the other one. As we can observe, regardless of the shift in scale and view point, the Harris-Laplace detector locates the same local features from images.

Figure 5. Harris-Laplace detection results, from [10]

3. Reference

[1] RWTH Aachen, computer vision group

[2] CS 376: Computer Vision of Prof. Kristen Grauman

[3] S. Seitz
[4] Forsyth & Ponce

[5] Prof. Svetlana Lazebnik

[6] Prof M. Irani and R. Basri

[7] Prof. Li fei-fei

[8] Denis Simakov

[9] Alexej Efros

[10] Prof. Krystian Mikolajczyk

[11] SIFT

[12] Saad J Bedros

[13] SIFT2

Any corrections, suggestions, and comments are welcome

--

--

--

This publication is for organizing and posting what I’ve studied in scope of CS and Mathematics

Recommended from Medium

I am the first Machine Learning Engineer, now what?

Image Classification made easy

Introduction to Reinforcement Learning for Beginners

Bayes’ Theorem — Some Perspectives

Classification of MNIST data with 100% accuracy

Literature Review: MultiNet

Association rule learning: A brief overview [Part-1]

A explicação mais simples sobre machine learning que alguma vez lerá

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
jun94

jun94

More from Medium

How Computer Vision Can Help Robots Navigate the Real World

An Exploration into DETR: End-to-End Object Detection with Transformers

AI Tennis Ball Bounce Detection

Image Super-Resolution in Python