Image Registration: From SIFT to Deep Learning

How the field has evolved from OpenCV to Neural Networks

Written by Jeremy Joslove & Emna Kamoun

Read the original article on Sicara’s blog here.

Image Registration is a fundamental step in Computer Vision. This article presents OpenCV feature-based methods before diving into Deep Learning.

What is Image Registration?

Image registration is the process of transforming different images of one scene into the same coordinate system. These images can be taken at different times (multi-temporal registration), by different sensors (multi-modal registration), and/or from different viewpoints. The spatial relationships between these images can be rigid (translations and rotations), affine (shears for example), homographies, or complex large deformations models.

Image registration has a wide variety of applications: it is essential as soon as the task at hand requires comparing multiple images of the same scene. It is very common in the field of medical imagery, as well as for satellite image analysis and optical flow.

CT scan and MRI after registration

In this article, we will focus on a few different ways to perform image registration between a reference image and a sensed image. We choose not to go into iterative / intensity-based methods because they are less commonly used.

Traditional Feature-based Approaches

Since the early 2000s, image registration has mostly used traditional feature-based approaches. These approaches are based on three steps: Keypoint Detection and Feature Description, Feature Matching, and Image Warping. In brief, we select points of interest in both images, associate each point of interest in the reference image to its equivalent in the sensed image and transform the sensed image so that both images are aligned.

Feature-based methods for an image couple associated by a homography transformation

Keypoint Detection and Feature Description

A keypoint is a point of interest. It defines what is important and distinctive in an image (corners, edges, etc). Each keypoint is represented by a descriptor: a feature vector containing the keypoints’ essential characteristics. A descriptor should be robust against image transformations (localization, scale, brightness, etc). Many algorithms perform keypoint detection and feature description:

• SIFT (Scale-invariant feature transform) is the original algorithm used for keypoint detection but it is not free for commercial use. The SIFT feature descriptor is invariant to uniform scaling, orientation, brightness changes, and partially invariant to affine distortion.

Read the full article here.

If you want to learn more about OpenCV, check out our article Edge Detection in OpenCV 4.0, A 15 Minutes Tutorial.