The science behind photogrammetry.
The word photogrammetry is derived from two parts: photo — meaning “picture”, and grammetry — meaning “measurement”. It can be defined as a method for obtaining information such as length, area, volume etc from images. It is broadly a three-step process.
- Structure From Motion (SFM)
- Multi-View Stereo (MVS)
- Images rectification
These processes can be further divided into following sub-processes.
1.Image Feature Extraction — Each image has a collection of unique features which differentiate it from other images. These are known as keypoints. Key points from each image are extracted using automatic computer vision algorithm (SIFT, BRISK, etc). These features consist of Building’s corner, Roads, edges, etc.
- Generally, images with good texture variation have>40,000 features. It can be easily understood why photogrammetry performs poor in areas of low texture variation like water bodies, dense forest, sand, sky etc. Keypoints extraction becomes difficult in textureless surfaces.
Image shown on left has three scenes Land/Bridge, Sand, Water regions. Each circle represents a unique feature.
1. The density of features in the leftmost and Bridge region is very high since it has a lot of edges, colour changes, etc
2. Very few features are detected in sand and water regions due to textureless surfaces.
2. Features Matching — Extracted features are then searched in the nearby images and matching is performed. Using GPS data to search relevant images makes the matching process much faster and accurate. From matched features, fundamental matrix is derived and the relative position between two cameras is estimated. Techniques like Flann is often used for searching and matching.
3. Bundle Adjustment(BA) — Relative position estimated from the fundamental matrix is prone to errors. BA is used to simultaneously refine the 3D coordinates (Lat, Long, Elevation), orientation’s parameters (Yaw, Pitch, Roll), and the optical characteristics (distortion parameters) of the camera(s) employed to acquire the images. BA is a non linear iterative optimization process where the objective function is Mean Reprojection Error (MRE) and parameters are the position, orientation and camera’s distortion coefficients.
- Geotag data stored in images are used to georeference and scale the model.
4. Depth Map estimation and Point Cloud generation — Depth value is estimated for every pixel in an image using Multi-View Stereo algorithm. MVS algorithms are able to construct highly detailed 3D models from structured images. So the output of SFM will act as input to MVS algorithm. It will output the depth map corresponding to every input image.
Individual Depth Map is fused together with neighbouring image’s depth maps to obtain the 3D point. These points are often called as the dense point cloud. It may even consist of >1crores points for a relatively small area.
5. Digital Elevation Model (DEM)— 3D Points are triangulated and grided in 2.5 Dimension space to create 2.5D Digital Elevation Model (Raster). Every pixel in raster has latitude, longitude and elevation information. Interpolation techniques like IDW are often used to do 3D point cloud to 2.5D grid/raster conversion.
6. Orthomosaic (HD Map) — Orthorectification of each photo is done using DEM. Orthorectification step involves creating a visibility or occlusion map with respect to each image. These maps tell us which pixels are visible or occluded(not visible) from a particular image. Only visible pixels are then selected and colour values are extracted. These orthorectified and occlusion free photos are mosaiced together to create a large HD Map.
Post-processing techniques such as Image Blending, Color/Contrast adjustment are used to remove seamlines present at the boundary of images, to make colour uniform and free from artefacts.
If you want to get detail explanation about deliverables, it can be found here.
About Indshine
Indshine provides enterprise drone solutions to highways, railways, mining, smart city, power transmission, forestry etc. Please visit Indshine to know more. If you have any query or want a demo, kindly drop us an email at info@indshine.com.
