Getting started with Photogrammetry — with a Smartphone camera [2019]

Capturing the world in a Volumetric data format

Published in

Realities.io

14 min readOct 18, 2017

With the recent advancements in computer vision, spatial computing devices (VR/AR headsets), high quality cameras in our pockets, and open sourced software, a 3D capturing method called Photogrammetry is becoming more accessible to game developers, photographers, as well as VFX artists.

You’ve probably seen all the cool videos, played games with hyper-realistic assets (like, or even visited Photogrammetry locations in VR. So, here’s how to get your feet wet into capturing Photogrammetry with your smartphone and free software!

Cologne Cathedral in Realities: https://www.youtube.com/watch?v=fnXehd7oxto

Pre-Requirements:

Camera (I’ll be using an iPhone 7 Plus for this tutorial)
a PC w/ Nvidia GPU (highly recommended), 16GB RAM (32GB+ recommended)
Photogrammetry software: Meshroom (free and open-source!)
3D editing software: Meshmixer (also free!)
Patience (and lots of it)

Choosing the subject

You can divide Photogrammetry into a few different of genres based around subjects, including interiors captures, building exteriors, terrain and grand landscape, macro Photogrammetry, people scanning, shoe scanning, tree stump scanning, and etc. Check Sketchfab’s Photogrammetry section for inspiration.

I specialize in interior and exterior captures, which is what we’ll also focus on in this tutorial (which also don’t require extra equipment like lighting, turntables, or drones).

There isn’t any real difference in how the subjects get processed and created into 3D, but how you take pictures.

Exterior captures (Capturing houses or singular objects)

Church in Croatia: https://sketchfab.com/models/22550a6bbe5c4ad4869389f2c4ae7566

Exterior captures can be somewhat easier to begin with purely because they’re harder to mess up. You have a subject that you can walk (or fly) around and take a overlapping photos along that path.

The scale of the subjects can vary. You can capture everything from the exterior of a building to something as small as a pinecone. It all depends on what cameras you have access to.

Interior subjects (inside out photography)

Interior capture of a train cabin. https://skfb.ly/6Rt7S

Interior captures such as bedrooms may seem easy, but typically are one of the more difficult things to capture and process cleanly. White, featureless, clean walls can make alignment difficult and depthmaps more noisy.

When it comes to capturing the interiors of environments, the best tools at your disposal will be:

A camera like a DSLR that can handle lower lighting situations (new smartphones can be decent at handling these situations as well)

a WIDE angle lens (ideally between 12–20mm)
a tripod (if the space is actually really dark and you need longer exposure times).
TONS of overlapping pictures

For a more in-depth interior capture tutorial, read my Getting Started with Photogrammetry Part 2.

Capture Day

Now that you’ve decided what you want to capture, let’s start with planning the capturing method.

For this example, I’ll be shooting with my iPhone 7 Plus using the stock Camera app (12MP camera with ƒ/1.8).

Start at a position with the full subject in view, walk around your subject, and take a photo at every step. Be sure to stop to take each photo to ensure sharpness and reduce rolling shutter issues.

You can download the 97 images to work with the same image dataset.

Keep the lighting consistent

Minimize the capture time to keep the lighting even between the first and the last photo for your data set. This wasn’t an issue for our brick fireplace tutorial as the total capture time was ~2 minutes, but can be an issue for much larger projects (1hr+ capture times).

This scan of the Geghard Monastery was done over the course of a few hours. As you can tell, the sun moved, and created two different angles of shade. I should mention, there are ways of fixing this, and it requires turning off specific images being using in texturing if the lighting/exposure is different than the rest.

Two different angles of shade over the course of an hour

Remember, shadows move slower when the sun is directly vertical(~12:00 PM) than at dawn or dusk. Ideally, a cloudy day gives you more leeway to capture over a longer period of time without noticeable harsh shadows moving.

Most Photogrammetry guides will recommend that you capture only on cloudy days… which can be useful if you want to capture smaller subjects like rocks and wood to turn into a PBR material and use as an asset in a video game.

However, that is not how I use Photogrammetry, especially for the subjects that I capture (check out my work on Sketchfab). I see this technology as an natural extension of Photography/Cinematography and apply the same principles to capturing scenes in the most beautiful lit settings.

My final scenes are almost never lit, as the normals you get from Photogrammetry are typically noisy and not ideal to be lit. If you have the scene be unlit in engine or in Sketchfab, your textures will look much more photorealistic than anything a standard shader will provide.

the natural sunset lighting really make this scene pop.

Two important details to remember:

Capturing for alignment: To make sure that you all your photos align as one 3D model, your pictures need to have a ton of overlap (at least 50%). It’s best to use a non-fisheye, wide angle lens (12mm –18mm) to reduce the amount of total overlapping photos you need. It’s better to stand far and capturing beyond the subject that you’re interested in.

If capturing a large area (like a house with rooms), its best split up the capture of each area as it’s own “run” which all share a common starting and ending point (somewhere in the middle of the house). You can think of each run as “closing the loop” so that the Photogrammetry software can recognize that how these other rooms are connected to each other.

Capturing for texture: The other important side of the capture method is to ensure you have close-up photos of the subject, to get those fine textures and geometry integrated into your scan. It’s important to take pictures between your overview “alignment” photos and these close-up “texture” photos so the software can make sense of the camera movements.

Photogrammetry Processing: Meshroom

For this tutorial, we’ll be using Meshroom. It’s an open-sourced Photogrammetry processing software based on the Alicevision framework. The teams behind the software have made significant performance updates since its inception in 2018, which is why we’ve updated this tutorial to base it entirely around this free software.

It’s worth mentioning that at realities.io, our go-to Photogrammetry processing software is Reality Capture, primarily because our image datasets can be large (2,000–20,000) and heavy (42mp per image), so reducing processing time is absolutely essential for our project turnaround times. However, we’re quite impressed with the progress made with Meshroom as well as its flexibility for different workflows and pipelines and are eager to see what developers can do with it.

Transfer images to PC

For iPhone + Mac users, the Airdrop workflow is quite decent. Select up to ~150 images at a time, and airdrop them to your Mac.

For PC users, we’ve used Google Drive to upload and transfer images. Android phones do have an advantages of granting root access to your files via USB, so transferring images without needing to upload to a cloud location is great.

Finish up by making a main project folder, and a subfolder called Images that contain your image dataset.

The simple 1-step, full Photogrammetry processing button!

If you’d like a quick, no hassle result of your work, you can import your images and press Start and Meshroom will align, mesh, and texture the capture.

This works great, but typically the end result will be much higher poly than you’d need.

We recommend adding a MeshDecimate node to decimate the mesh to a usable poly count.

right click in the Graph Editor, select MeshDecimate.
right click on the link between MeshFiltering and Texturing nodes and click Remove.
Click on MeshDecimate and in its Attributes, change the Max Vertices setting to 50,000 (adjust this amount up if you have a bigger scene that needs more dense mesh).
Connect the MeshDecimate node to MeshFiltering and Texturing as seen bellow.

adding a MeshDecimate node

Right click on Texturing node, and select Compute.

Once it’s done, double click on the Texturing node to load it into the 3D Viewer.

You can also hide the StructureFromMotion point cloud and cameras by clicking on the eye icon to get a better view of your textured mesh

To export the textured mesh, right click on the Texturing node and select Open Folder. You can also find it in this directory:

“…\[PROJECTFOLDER]\MeshroomCache\Texturing\…”

This files are what you want. OBJ, MTL, and the textures

The mesh will most likely have a lot of polygons, a bunch of the surrounding reconstructed, and a few 8k textures.

HOWEVER, it’s common that there are parameters you can optimize to get better camera alignment, better mesh quality, and better texture.

The following steps will go through various nodes and settings that can help you achieve better results.

The Detailed Photogrammetry processing steps

Nodes based iteration system

Before we begin jumping into tweaking settings, you should know how Meshroom’s node based system is fantastic for iterating on tweaking settings, processing, and comparing the results.

For any node, if you’d like to change a setting to see how it affects its results, you can right click, and select the “Duplicate >>”. This will duplicate that node and every node after (which is useful because it’s hard to remember how to link all the nodes together and in which order).

Duplicate a set of nodes to compare results of different settings

You will see how we’ll use this iteration system for getting the best results of our mesh for our needs.

Image and camera properties

Step 1: Drag and drop the images into the Images sidebar.

Make sure the Meshroom is detecting the correct camera and lens properties by hovering over the green icon on the images. If you’re using a common phone or camera, you should be just fine.

If not, you can look up your lens and sensor parameters and manually input them [guide].

confirm the make and model of your camera

Feature matching and Camera alignment

The first step to Photogrammetry processing is feature extraction and camera alignment.

Meshroom loads with all the Photogrammetry nodes you need. Don’t be put off by how many nodes there are. You don’t really need to know what each does to be able to able to get good results. But with this tutorial, you’ll understand what each “cluster” of nodes do.

I spaced out the alignment nodes cluster for clarity.

For simple subjects like the brick fireplace subject we picked, where the subject is not reflective, the surrounding is visible and full of features, and I systematically moved between each image and ensured overlap between each image, we should be confident that the images should all properly align.

Step 2: Right-click on the StructureFromMotion node and click Compute. Meshroom will process all the nodes up until that node.

If successful, you should have all your images align (in my case, 97 images) and produce a sparse point cloud.

If you have less than 300 images, Meshroom recommends that you increase the FeatureExtraction node “Describer Preset” to High from Normal. All this means is that the feature extraction algorithm will find and use more features per image to find precise alignment of the cameras in the scene (which also means it will take more time to process, but for small datasets, it’s not very significant).

More features, mean more chance of photos aligning

If you’re having alignment problems,

In FeatureExtraction, increase the Describer preset to High or Ultra.
In FeatureExtraction, turn on Guided Matching setting.
In the FeatureExtraction, FeatureMatching, and StructureFromMotion nodes, change the Describer Types to AKAZE (default: SIFT). Meshroom documentation says “it may improve results from some surfaces like skin as it is more affine invariant than SIFT and can help recover connections when you have not enough images in input”.

Depthmaps and Meshing

The next step after camera alignment is to generate depth maps and create a mesh (the geometry of the Photogrammetry scene).

NOTE: Generating DepthMaps ONLY works for Nvidia equipped PC’s, so for everyone else, you’ll have to use Draft Meshing and skip generating Depthmaps.

Again, I’ve arranged the nodes in clusters for the sake of this tutorial.

Step 3: Right click on Meshfiltering and click Process. Go and have some coffee, this step will take the longest.

Once it’s done it should look like this:

As you can see, it produced a mesh with 3.9mil polygons and a fair bit of the surrounding environment (which we don’t really need).

Improving Mesh Quality

If you’d like to improve the meshing quality of the subject, there are a few settings to tweak.

To decrease meshing the surrounding,

in the Meshing node, you can increase Min Observations Angle for SfM Space Estimation much higher to discard meshing the surrounding

To have a denser mesh reconstruction (higher poly count),

in the Meshing node, you can increase Max Input Points and Max Points (this is limited by your PC’s RAM. 16GB RAM users might need to lower these parameters)
In the DepthMap node, you can decrease the Downscale setting from 2 [default] to 1 (note: this will increase processing times by 4x)
If you’re getting holes in your mesh, in DepthMapFilter, you can reduce the Min Consistent Cameras to 2 and Min Consistent Cameras Bad Similarity to 3
if your mesh is still noisy generally more noisy than you’d like, you can increase the MeshFiltering Smoothing iteration amount.

However, after some iteration and tweaking settings, I wasn’t getting a mesh that looked all that different from the default settings. This might be tied to the limited quality of my camera and the small size of the scene.

The biggest improvement in mesh quality I observed was setting the DepthMap downscale factor to 1 (default 2).

Mesh retopology and quick editing / crop

It’s recommended to edit the higher poly mesh to have a “master mesh” (the result before MeshDecimate), so that when you create lower poly derivatives, you don’t need to redo all your work.

Step 5: Launch Meshmixer to edit the mesh

Drag and drop the mesh you’d like to edit (recall that you can right-click on the last node you processed and select “Open Folder” to find your mesh)
If there are floaty triangles that you’d like to get rid of, select the “Select” tool (shortcut “S” key), double click on the largest component, then press “I” to invert your selection to everything that’s not connected to the mesh. (note there, the MeshFiltering node also has an option to “Keep Only the Largest Mesh” which I could’ve used)

Select the model then inverse to select floaty bits.

Now let’s crop out the surroundings with a clean cut. Select Edit > Plane Cut. Left-click and drag a line to set the location of the plane cut. Flip the direction of the cut if need be (as seen in the bellow).

Cropping the surrounding using “plane cut”.

There might be some parts of the mesh you’d like to smooth out. I use the Sculpt>Robust Smooth tool as seen in the gif bellow.

If you’re happy with your edits, export is as an OBJ to any accessible folder (doesn’t need to be your Meshroom project’s directory) and use this mesh as your “Master Mesh”.

Re-import mesh into Meshmixer

We’re almost there! We just need to decimate and texture, and we’ll be done.

The mesh re-import process can be done on any Node that accepts meshes as an input. You just need to disconnect the link from the previous node so you can type in the directory of the mesh you’d like to import.

Step 6: re-import mesh into Meshroom

Create a MeshDecimate node and connect it between Meshfiltering and Texturing
Now, to import the Meshmixer edited “master mesh”, disconnect the MeshDecimate Input link, select MeshDecimate, and drag and drop your edited OBJ onto the Input Mesh text field (as seen in the gif bellow).
Select MeshDecimate, and change Max Vertices to 50,000 (equivalent to 100k poly)

Texturing Settings

Final Step: Select Texturing node, and click compute.

The default Texturing node settings will produce a high quality result, as seen in the screenshot bellow.

The problem is, textures take can take lot of space, as this is 43.6 MB

Default texturing settings. [Downscale 2, four 8k textures, Basic unwrap]

To meet Sketchfab’s basic account’s upload limit (50mb), I changed the settings to the following, totaling 23.9 MB.

[EDIT: I originally wrote this thinking the limit was 30 mb, not 50. However it’s still useful to find ways of minimizing file space while minimizing reducing quality]

Texture Downscale: 1 (so it uses the full resolution of the images)
Texture Size: 4k
Unwrap method: LCSM (which limits to 1 texture map)

Downscale 1, one 4k texture, LCSM unwrap

Best Texturing settings:

Downscale: 1 (default is 2)
Unwrap: Default (will produce multiple textures)
Max number of Images For Fusion: 3 or bellow (the less images per fusion, the sharper the image, but there’s a tradeoff if certain areas dont have enough images)

Let’s do a quick comparison between a few texturing settings.

Downscale 1, one 4k texture, LCSM unwrap — 23.9 MB

[Default] Downscale 2, four 8k textures, Basic unwrap — 43.6 MB

Downscale 1, four 8k textures, Basic unwrap (sharpest) — 113 MB

You’re done! Congrats

To export your textured mesh, right-click on the Texturing node and select Open Folder. Your Photogrammetry scan is the OBJ, MTL, and all the PNG texture files.

Publish your work

Now that you’re done, you want to do stuff with your Photogrammetry scans right? You can upload to Sketchfab to show off and share, you can import it into Unity or Tiltbrush, and even 3D print it!

Downscale 1, one 4k texture, LCSM unwrap — 23.9 MB

Preferable Sketchfab 3D Settings for unlit Photogrammetry

Shading: Shadeless
Vertex Colors: off (typical for Metashape/RealityCapture uploads with colorized verts)
Add a Ground Shadow > Baked AO for better framing
Straighten model till it’s straight
Pick a nice perspective and “Save View” as the model’s thumbnail and its starting position

For a more in-depth article about optimizing photogrammetry scans for Sketchfab, read my other t