Creating Your Own Digital Twin: Guide Through the Pipeline

If you’ve ever wondered how to bring an object into the digital realm, you’re in the right place. In this comprehensive guide, I will walk you through the essential steps of the asset creation process.

I’ve designed this guide to provide a fundamental pipeline for creating your digital twin. Let me share what has worked best for my projects.

You’ll need a basic understanding of 3D concepts and terminology because I won’t delve deeply into explanations of specific processes and terms like retopology, UV Tiles, normal maps, diffuse maps, and more.

The Pipeline

Over the past few months, I’ve been deep-diving into the world of 3D asset creation, focusing on unique real-world objects. With this in mind, I’ve developed a reliable pipeline that I use for every asset I encounter.

Here’s my digital twin creation pipeline:

1. Scanning
2. Reference Images
3. Processing the Scan
4. Retopology
5. UV Unwrapping
6. Bake Texture Maps
7. Export Model

Now I will go into more detail with every step.

1. Scanning/Photogrammetry:

• Purpose: Nothing beats a real object scan for accuracy compared to modeling from reference pictures or relying on memory.
• Techniques: My preference is photogrammetry, but you can also use a laser scanner or even a mobile phone. While AI-assisted scanning is gaining popularity, photogrammetry and laser scanning remain the most dependable and researched methods.
• Be Mindful of the Scanned Object:
  It’s crucial to consider the type of object you’re scanning. Scanning a non-reflective object, such as a tree or a stone sculpture, is relatively straightforward.
  However, challenges arise when dealing with reflective, transparent, or glossy objects like cars, glass, or monitors. These situations demand additional steps to obtain a usable scan.
  Likewise, be cautious when dealing with featureless objects like a plain table or a wall with a uniform white color. Software may struggle to capture these objects accurately, potentially resulting in distorted or jittery geometry.
• Goal: Create a reference mesh with correct proportions and a usable diffuse texture. A successful scan can also serve as the base for a normal map.

2. Shoot reference images:

• Purpose: This step provides reference material for later stages.
• Techniques: Capture photos from various angles or create a short video to observe how the object reacts to light. This is especially useful for areas where the scan may have missed details or textures.
• Goal: Gather reference images showcasing the object’s material properties.

3. Process the Scan:

• Convert the scan images into a 3D mesh using software like Reality Capture or Meshroom. Extract a high-quality model and the generated diffuse texture.
• Note: While Meshroom is free, Reality Capture often produces superior results but is a paid tool.

point cloud from images -> final mesh with textures

• Photogrammetry can be challenging and involves fine-tuning. Seek additional information from tutorials or program documentation to avoid common pitfalls.

4. Retopology:

• Purpose: Create a high-quality mesh with reduced complexity for better performance in rendering and real-time applications.
• Technique: Import the scanned mesh into your 3D software (e.g., Blender) and create a low-poly version with a focus on good edge flow. The polygon count should typically range from 5,000 to 100,000 vertices, depending on your project’s requirements.
• Goal: Achieve a manageable model with clean topology, which saves time when creating UVs and textures.

scanned mesh with 5.000.000 vertices -> retopologized mesh with 4.582 vertices

5. UV Unwrapping:

• This step is crucial for PBR material creation. Ensure that UV Islands fill at least 70% of the UV Tile for optimal texture utilization.
• Maintain consistent texel density across all UV Islands. If you’re not using the asset only for a VFX or close-up shot, keep all Islands on one UDIM.

UV with same texel density for all faces filling ~70% of the UDIM

• Goal: Prepare the asset for applying colors and textures.

6. Bake Texture Maps:

• We have a low poly asset that has good UVs. But it has no texture information and we already have the texture information from the scanned asset. So we can bake the texture of the scanned object onto the optimized aset.
• Technique: Utilize software like Blender or Substance Painter to perform this step. You can also create height and normal maps from the detailed scan geometry for PBR material creation.

7. PBR Material Creation:

• Purpose: Here you will give the asset the final touches so that it reacts correctly to the light just like the real one does.
• Technique: Utilize the baked diffuse, normal and height map. Create the roughness, metallic, and translucency map as needed. Substance Painter is a suitable tool for this step. You can also use the reference images taken in Step 2.
• Goal: Achieve a finished asset that behaves realistically under different lighting conditions.

some renders I did recently with digital twins

8. Export: Congratulations, you have a digital copy of your real object and you can do whatever you want with it! Export it into your game, or print it out to have a 3D-printed copy.

comparison between the real object and the digital twin

I hope this guide gave you a better understanding of the basic steps needed to start with your very own digital asset. If you have any questions, or when I should go into more detail with a certain step, let me know!