Part 3 — Lucas-Kanade Optical Flow
This is Part 3 of my series on GPS Denied Navigation
- Part 1 — Using OpenCV to Track Features
- Part 2 — The Math Behind Optical Flow
- Part 3 — Lucas-Kanade Optical Flow
Here is an LK optical flow determining the x and y velocity of the aircraft. The code to generate this is at the end of this post.
In Part 2 we discovered the following equation describes a tracked pixel with unknown velocities u and v as spacial motion induced by the camera.
There are two unknowns and a single equation which means this is really tough to solve. To get over this, instead of looking a single pixel’s movement over time, we want to track a collection of colocated pixels assuming they have the same u and v. Now we can capture multiple equations and solve for u andv with least-squares.
Below is a downward looking camera with a window where we’ve identified a good collection of pixels for tracking motion of the camera in the x and y axes.
When eigenvalues are small, there is no gradient in all directions, it is likely we are looking at pixels over a flat area that don’t show any correlation, like the river in the image above. When eigenvalues show a large ratio, 1 strong, 1 weak gradient, it means we’re looking at an edge, eg the riverbank. This collection is only useful for determining either u or v but not both since we can clearly see when we move laterally, but if we move up the edge, from the camera’s point-of-view we might not be moving at all.
Where we find large eigenvalues, thats where we know it is a highly textured region of the image that serves as a good collection of pixels to track.
Similar to the Shi-Tomasi feature detector we used in Part 1, the Optical Flow algorithm has many tunable paramters. We’ll be using cv.calcOpticalFlowPyrLK which uses the Lucas-Kanade method.
