theta360.guide
Published in

theta360.guide

Photogrammetry with RICOH THETA and 360° Images

Create 3D models of 360 images and free open source software

3D Scene Automatically Created Using RICOH THETA Images

We saw a huge expansion of 360° camera use in 2021, driven by rapid growth in virtual buying experiences for cars, houses, and other types of spaces that people can go into. In addition to building virtual buying processes, 360 cameras proved to be excellent to support complex workflows in the construction industry.

At the very cutting edge of 360° image use is photogrammetry and AI to identify objects and measure distances. Photogrammetry is the process of converting RICOH THETA images into 3D models. The models are then used for analysis. To illustrate what photogrammetry is, let’s first look at how the open source Meshroom software is used with normal 2D images.

Step 1: Take Many Pictures of the Same Scene from Different Angles

Before we start using a RICOH THETA camera, I’ll first create a model of an owl statue with normal 2D pictures.

Step 2: Create Point Cloud

The images can be converted into common points by using software to compare the different images. The point cloud below only takes a few minutes to create on my laptop.

Step 3: Apply Colors

Step 4: Apply Textures

The model can be rotated and inspected.

Using 360 Images

RICOH THETA images offer the potential to create full scenes from 360° images.

Although the models can be impressive and fun to render, there was quite of a bit of artifacts in my tests with Meshroom. It’s likely that more expensive commercial software would produce better results. You can try this yourself to understand photogrammetry, but Meshroom may not be usable commercial models.

Here’s a closeup of a watercan which illustrates some of the problems I encountered.

As this is my first time using Meshroom, I can likely improve the model with practice. I attempted to focus on the watercan, but ran into issues with the conversion from equirectangular to 2D images.

Position of RICOH THETA camera in test

I was able to create a more reasonable model of a teapot.

Using colors from RICOH THETA V Images

Shots taken with THETA V. Equirectangular image is split and only the teapot portion is used.

Equirectangular conversion to 2D using split method
Finding position of camera
Model made with RICOH THETA images before texture is applied

Process to Use RICOH THETA Images with Meshroom Open Source Software

Disabling topBottomCorrection and setting Exposure to Manual

This step may not be necessary, but I decided to disable topBottomCorrection and auto exposure to get more consistency in the input images prior to splitting the equirectangular images into separate 2D images.

.\opptheta_windows_0.3.1.exe getOptions --options=exposureProgram _topBottomCorrection
[exposureProgram, _topBottomCorrection]
{
"name": "camera.getOptions",
"results": {
"options": {
"exposureProgram": 1,
"_topBottomCorrection": "Disapply"
}
},
"state": "done"
}

Split 360 image into 4 Parts

Using aliceVision_utils_split360Images.exe

split each of the 11 images into 4 segments. The help provides the syntax.

Usage:This program is used to extract multiple images from equirectangular or dualfisheye images or image folder
AliceVision split360Images:
Required parameters:
-i [ --input ] arg Input image file or image folder.
-o [ --output ] arg Output keyframes folder for .jpg
Optional parameters:
-m [ --splitMode ] arg (=equirectangular)
Split mode (exif, equirectangular,
dualfisheye)
--dualFisheyeSplitPreset arg (=center)
Dual-Fisheye split type preset (center,
top, bottom)
--equirectangularNbSplits arg (=2) Equirectangular number of splits
--equirectangularSplitResolution arg (=1200)
Equirectangular split resolution
--equirectangularDemoMode arg (=0) Export a SVG file that simulate the
split

Split results in 44 files.

aliceVision_utils_split360Images.exe -i C:\Users\craig\Pictures\THETA\sc2_pillow -o C:\Users\craig\Pictures\mesh_test\pillow --equirectangularNbSplits 4
Program called with the following parameters:
* dualFisheyeSplitPreset = "center" (default)
* equirectangularDemoMode = 0 (default)
* equirectangularNbSplits = 4
* equirectangularSplitResolution = 1200 (default)
* input = "C:\Users\craig\Pictures\THETA\sc2_pillow"
* output = "C:\Users\craig\Pictures\mesh_test\pillow"
* splitMode = "equirectangular" (default)
* verboseLevel = "info" (default)
[11:51:31.093598][info] Find 11 file paths.
[11:51:33.697260][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012325.JPG successfully split
[11:51:36.122197][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012326.JPG successfully split
[11:51:38.481228][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012327.JPG successfully split
[11:51:40.954926][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012328.JPG successfully split
[11:51:42.709119][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012329.JPG successfully split
[11:51:44.258744][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012330.JPG successfully split
[11:51:45.755567][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012331.JPG successfully split
[11:51:47.225788][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012332.JPG successfully split
[11:51:48.671340][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012333.JPG successfully split
[11:51:50.121590][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012334.JPG successfully split
[11:51:51.630634][info] C:\Users\craig\Pictures\THETA\sc2_pillow\R0012335.JPG successfully split

Using Non-360° Camera to Compare Models

Using a Google Pixel 4 with auto-exposure off.

Resources

  • https://www.researchgate.net/publication/325471245_CAN_WE_USE_LOW-COST_360_DEGREE_CAMERAS_TO_CREATE_ACCURATE_3D_MODELS

Additional Scenes

3D Reconstruction of Balcony

Summary

Generating a full scene point cloud with Meshroom from RICOH THETA images can be a lot of fun, but there are many artifacts in the model. The model can likely be improved with more knowledge of Meshroom, which is a lot of fun to experiment with. Another great experiment is to try software that can directly handle 360 images to recreate a complete scene with Agisoft PhotoScan and Pix4Dmapper.

As this is my first attempt at using photogrammetry software, I would appreciate any advice.

Other Tips

Location of split utility from Meshroom.

Directory: C:\Users\craig\Downloads\Meshroom-2021.1.0-win64\Meshroom-2021.1.0\aliceVision\binMode                 LastWriteTime         Length Name
---- ------------- ------ ----
-a---- 12/31/2021 7:40 AM 136704 aliceVision_utils_split360Images.exe

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Craig Oda

Craig Oda

open source advocate, writer, Flutter developer, father, husband, fly fisherman, surfer