Overview of Drone Photogrammetry Software
Drone Photogrammetry has become a common place in the drone world. Whether you are mapping a site or a building, a 2D map or a 3D model can be a great way to digitize infrastructure and understand the condition, quantity and status of objects on the job site.
Over time, a number of different software packages have become available to help with drone photogrammetry. You give them a set of 2D images, and hit the 'Process' button and they give you back a 3D model, (albeit a couple of hour later).
But how does drone photogrammetry actually work? What goes on under the hood and why does processing some maps and models take hours to accomplish?
We'll answer all of these question in our post below.
Data Pipeline
In order to convert a set of 2D images into a 3D model, drone photogrammetry software typically implements a set of operations on the input data (drone images). These operations are organised in the form of back-to-back steps where the output from one step is the input to another step. This set of back-to-back steps is appropriately called a Data Pipeline.
A Data Pipeline can involve a number of different back-to-back steps. Let's take a look at the common ones below.
Step 1: Feature Extraction
The goal of this step is typically to find out "key features" (group of pixels) in the images which are invariant to the changing camera positions i.e. they look the same from different points of camera view.
Therefore, good features in the scene typically have similar descriptions (mathematical structures) in all images.
💡 This is one of the reasons why drone photogrammetry requires capturing the same subject from many different points of view.
There are many techniques used in software to extract these "key features". One of the common methods is SIFT (scale-invariant-feature-transform) but other techniques can also be employed.
credit: researchgatenet.com
Step 2: Feature Matching
The goal of this step is to match the "key features" (found in Step 1) across different image pairs. This is done by taking every feature X, and then comparing all the other features in other images with feature X to see if the same feature can be found in other images. A quality score is calculated using a wide range of software techniques and image pairs with the same feature are found.
credit: researchgatenet.com
💡 This is one of the reasons why drone photogrammetry fails to reconstruct images with recurring patterns. With recurring patterns, you risk too many features looking the same across images!
So if you are collecting data for photogrammetry, a useful tip here is to avoid patterns and get enough different looking features in your dataset from many different points of view.
Step 3: Structure From Motion (Sparse Point Cloud Reconstruction)
The goal of this step is to reconstruct a 3D point cloud from 2D points. A 3D point cloud is basically a set of points with estimated pose (position and orientation) information attached to them.
From steps 1, 2 and 3 above, we have been able to identify a set of features (group of pixels) in images and also identify image pairs which hold these features. We can now use this information to infer the rigid scene structure (3D points with pose) and the internal calibration of all cameras.
The above image might look daunting but essentially, the software estimates 3D points using triangulation techniques and the "motion" of "key features" over time.
💡 In the image above, the red point and the purple point are features which have been found in 2 different images. The camera position and parameters are used to estimate the 3D position and orientation of the red and purple points.
Once this process is repeated for enough (but limited) number of points, we get what is known as a sparse point cloud, which can be visually inspected to see if the reconstruction is matching the target site or asset.
credit: researchgatenet.com
Step 4: Depth Map Estimation
Now that we have a sparse point cloud reconstructed, this means that the software has been able to put together a set of sparse 3D points with position and orientation information relevant to internal cameras.
This step of the pipeline focuses on calculating the depth value for each pixel in the point cloud. The depth value of every pixel is essentially the distance of every pixel from the internal cameras computed in the previous step.
Step 5: Dense Point Cloud (Mesh) + Texturing
Once depth maps have been successfully computed, the goal of this final step in the data pipeline is to create a dense representation of the scene (3D Mesh) and add textures to the 3D mesh.
You can think of this step as essentially "filling in" the sparse point cloud by using all the pixel information from the input images and utilising the full resolution of the input images.
The output of this step is a dense point cloud, giving us a 3D model of the target in its full resolution, with added textures.
This textured 3D mesh can then be exported into common 3D formats such as .obj and can be imported into 3D software for further processing.
And that's it! At a high level, these are the 5 main steps involved in data processing using photogrammetry software.
Summary
We hope this guide helps you understand the internal workings of drone photogrammetry software and the importance of collecting high-quality data for high quality results.
If you would like to learn more about Photogrammetry and how to collect high-quality data using Hammer Missions please see the post:
If you'd like to try out photogrammetry, you can download our free sample drone dataset over here.
At Hammer Missions, we develop state-of-the art software to help you capture and process photogrammetry data using drones.
If you haven't got a Hammer account as yet and would like to try Hammer Missions you can get started on our free trial.
To learn more about our enterprise solutions, including mission collaboration, data processing, and AI solutions, please contact us at team@hammermissions.com.
We look forward to hearing from you.
— Team at Hammer Missions