Optical Flow Estimation using C++
Optical flow describes the motion of objects, surfaces and edges given a sequence of images. It can be thought of as a vector that describes the displacement of each pixel in an image from one frame to the next. It is particularly important concept in computer vision applications, including video compression, object tracking, autonomous driving, and robotics.
In this article, we will implement an algorithm to compute the optical flow of two ordered images using Horn-Schunck method in C++.
Horn-Schunck algorithm minimizes a global energy functional in form of following:
This minimization problem is equivalent to the following partial differential equations system:
A finite difference discretization is applied to compute the derivatives. Discrete version of above equations system is
where Ix, Iy, and It are partial derivatives of the image.
This can be rewritten as the a linear system of equations Ax=b where A is (2*width*height x 2*width*height) and b is (2*width*height x 1) matrix that are computed from the image sequence. X is the array
which contains the optical flow field.
Thus, the optical flow algorithm can be summarized as below:
- Read image sequence
- Get image derivatives
- Compute A and b
- Solve linear system of equations
- Get u and v
Let’s start!
- Read image sequence
To read images, we will use CImg library from C++
2. Get image derivatives
We will stack two images into one 3D image, then we will use get_gradient method to compute gradients easily.
3. Compute matrix A and b
To compute matrix A and b, we will use discrete version of equations. We will compute coefficients for each i,j from NxN matrix (which is the shape of partial derivatives). For the first equation of the discrete equations, we will have the following linear system:
and for the second of discrete equations, we will have following linear system:
Stacking these two, we will obtain the matrix A.
Matrix b will have the form of the following:
4. Solve linear system of equations
After obtaining matrix A and b, rest is solving the system. We will use eigen library’s linear solver in this case.
5. Get u and v
After solving X, we will get u and v by simply extracting values row by row. Then, we normalize the vector field and quantize it. That’s it.
Let’s look at an example, giving the following image sequence
We will obtain the following vector fields u and v:
Full repository can be found in the link below:
If you have any questions or comments, I’ll be happy to answer them!
References
