Camera calibration. OpenCV on iOS.
Computer Vision & Robotics A-Z Guide.
Give us a message if you’re interested in Blockchain and FinTech software development or just say Hi at Pharos Production Inc.
Or follow us on Youtube to know more about Software Architecture, Distributed Systems, Blockchain, High-load Systems, Microservices, and Enterprise Design Patterns.
A camera has a lens. A lens can distort your image. But why and how? You can read this on Wikipedia and 1000+ photography web sites.
Our goal is to remove distortions from image result. We will deal with radial distortions in this article. Why it matters? Fisheye is a perfect radial distortion example — short focal distance lenses will always distort photos. Low-quality lenses loves to add distortion too. This type of distortion skew straight lines, so we should undistort camera output before we begin to make image processing. Surely if we care that the image is distorted. Usually, we don’t, but sometimes. Camera calibration will remove distortion programmatically for us.
PREPARE
There are two ways to calibrate a camera in OpenCV. Using chessboard pattern and using circles pattern. We will try with circles next time maybe because chessboard works fine for us. Simply open chessboard image on your monitor and take from 15 to 20 shots from various angles. You can use any chessboard you like. Just don’t use something really small. Perfectly visible, big chessboard image is what we need.
DO IT
OpenCV compensates distortion by generating appropriate distortion model. This is the first step. It should give us two matrixes — camera matrix and distortion matrix. Also, it returns a set of initial training images but with applied test grid on them.
We have 3 calls inside this function
- addChessboardPoints — will find all points on images and in 3D space.
- calibrate — this is camera calibration function. It returns an error in pixels and creates two matrixes — camera matrix and distortion matrix.
- getCameraMatrixAndDistCoeffMatrix — simple accessor to fetch both matrixes into corresponding variables.
Chessboard with a grid looks like this. This is not a photo of monitor taken already with a drawn grid. This is our initial training photo set.
addChessboardPoints
Let’s step down through this code. We initialize 3 arrays(say, vectors). One for image points, one for points in 3D space and on for all result images — that chessboards with a grid on it. Then we add indexes of each cell corner into 3D space point array. BoardCellsNum contains a number of rows and columns of our checker image minus 1. So we have 10 columns and 7 rows. BoardCellsNum is cv::Size(9, 6). Got it?
Next, we iterate over all the photos and grayscale them. Grayscale means — we work only with one channel to avoid meaningless computations.
Next snippet will find all corners, then get subpixel accuracy on edges, then draw a test grid and save all images into an array. A short explanation:
- CV_CALIB_CB_ADAPTIVE_THRESH Use adaptive thresholding to convert the image to black and white, rather than a fixed threshold level (computed from the average image brightness).
- CV_CALIB_CB_NORMALIZE_IMAGE Normalize the image gamma with equalizeHist before applying fixed or adaptive thresholding.
- TermCriteria Termination criteria for the function — we set max iterations to 30 and minimum subpixel accuracy to 0.1
- winSize Half of the side length of the search window. For example, if
winSize=Size(5,5), then an 11x11 search window is used
- zeroZone Half of the size of the dead region in the middle of the search zone over which the summation in the formula below is not done. It is used sometimes to avoid possible singularities of the autocorrelation matrix. The value of (-1, -1) indicates that there is no such a size.
And the last part of a function — we save a result into a global state.
CALIBRATE
Next part of our calibration module is a calibrator by itself
This function contains just one step. Calibration call which takes saved corners from the previous step and creates two matrixes — one is a camera matrix and another is a one-row matrix(say, vector) of distortion coefficients. It also generates rotation and translation vectors, but we will not use them.
SERIALIZE
Now let’s add a simple serialization to store both matrixes and reuse them for all future images. Both serialization and deserialization methods are pretty straightforward.
One more thing we need is to compute X and Y offset matrixes to move all pixels according to calculated values. We have two options
As you can guess from this screenshot we will use the second approach.
Thanks for reading!
