I’ve been working on an A-Frame hobby project Streetmix3D which creates a 3D version of streetscapes designed using the drag and drop builder Streetmix.net. (Streetmix started as a Code for America hackathon project and has been faithfully maintained as an open source project for the past seven years!)
The visual style of Streetmix3D started out with voxel stylized graphics created in Magicavoxel (left). I then experimented with using a photorealistic “custom photo” style texture I made using photogrammetry techniques and made into a simple repeating texture using Substance Bitmap-to-Material (right).
While I liked the look of the “custom photos style” (right) up-close, the “game ready” texture (center) looked better from a distance. I eventually found a reasonable quality output using a “Game Ready” style texture from Textures.com.
While these texture improvements are more details than voxels, they are simply repeating “image map” textures from a single JPEG or PNG. They do not exhibit real life lighting effects aside from built-in defaults of A-Frame and three.js.
Moving beyond 80s and 90s style graphics on the web
The evolution of Streetmix3D textures reminds me of the evolution from polygon 3D style games from the early 90’s to real time texture mapping in mid to late 90’s. When I was a kid I played a lot of racing games including the PC MS-DOS game Stunts (solid 3d polygons) and later the Need for Speed series (image map textures and sprites).
Now it’s time to try to take Streetmix3D to the next level out of the 90’s and into the new millennia of 3D graphics using Physically Based Rendering (PBR).
PBR with A-Frame and three.js
To learn about physically based rendering (PBR) and A-Frame I decided to start with a simple Project Goal: Display a plane in A-Frame that uses a PBR texture purchased from a third-party library.
I saw the best results using PBR with A-Frame and three.js with each of the following texture types to make the material look the best:
- Ambient Occlusion
- Image Map (aka Diffuse Map, Base Map)
- Normal Map
The scene will still render OK without all of these textures to varying degrees of success, depending on which are missing and what are your scene’s aesthetic goals.
Finding a PBR Texture from CGAxis
I was particularly interested in displaying a more realistic portrayal of the interplay between rail and asphalt for the tram and light rail lanes.
I found a beautiful looking “Road with Tram Rails” PBR Texture from CGAxis — it was only $5 and had everything I needed!* (*The metalness texture was missing at first but after a chat with support they quickly provided one.)
What’s inside of a PBR texture pack?
You may see varying files depending on the target platform and engine (Unity and Unreal seem to be the most common target platforms). Here are the files I found from the CG Axis “road with tram rails”:
- AO (ambient occlusion)
- Diffuse (aka base or image map)
- Reflection (aka specular)
To start making sense of these different files and their function I found this guide “PBR Guide Part 2” from Substance to be crucial helping me to understand the key parts of a basic PBR workflow. Highly recommended Sunday morning coffee reading!
Preparing textures for three.js
Discard — A-Frame is based on three.js which uses a “Metallic-Roughness” workflow and some of the files in the texture pack only apply to “Specular-Glossiness” workflow, so we can discard Glossiness and Reflection for this example.
Combine — Some of the texture images have only a single color channel (grayscale). These can be cleverly be combined into one image file by mapping each of the 3 grayscale images on separate channels for Red, Green and Blue. In fact we must combine these into one file for three.js.
Three.js expects Ambient Occlusion, Roughness and Metalness combined into one image file as RGB channels. You can combine this manually with Photoshop or a specialty tool like Substance Designer but that takes effort and those tools cost money, so I opted for a free command line solution instead!
For this I used ImageMagick — a versatile command line image manipulation utility. Install it using your method of choice, I used the homebrew command line utility for MacOS to install:
brew install imagemagick.
This installs a utility
convert that you can use to combine files together:
magick convert texture_ao.jpg texture_roughness.jpg texture_metalness.jpg -channel RGB -combine texture_combined.jpg
If one of the textures is missing, you can substitute it with black using this method but it will affect the quality of your final output:
convert r.jpg g.jpg -background black -channel RG -combine combined.jpg
Read more about
convert from the helpful ImageMagick documentation.
Displaying it with A-Frame
I could not find a way to define all these elements of PBR textures in A-Frame “out of the box.” I did find a great example implementation in the form of an answer to a Stack Overflow question How to Apply Environment Map to glTF in A-Frame written by Piotr Adam Milewski. He also provided a Glitch project proof of concept that renders a “Damaged Helmet” test object with an equirectangular environment map: https://stack-59625510.glitch.me/
I “remixed” Piotr’s glitch and replaced the glTF helmet with a plane geometry instead. This is practical for Streetmix3D roadway and in a more general sense to allow quickly experimenting with new textures before needing modeling software like Blender to apply it to a model.
I have renamed Piotr’s component to
envmap and created a new component
pbr that applies the specific textures to the plane material to support PBR. The syntax of the two components together are:
<a-plane pbr envmap></a-plane>
pbr component does the following:
- Load the textures from hardcoded URLs using
- Traverses through all the materials in the three.js object3d
- For each material: set the material’s maps to the loaded textures:
el.material.map = textureDiffuse;
el.material.aoMap = textureCombined;
el.material.metalnessMap = textureCombined; // can't set w/ A-Frame
el.material.roughnessMap = textureCombined; // can't set w/ A-Frame
el.material.normalMap = textureNormal;
var geometry = mesh.geometry;
geometry.addAttribute( 'uv2', new THREE.BufferAttribute( geometry.attributes.uv.array, 2 ) );
What does it look like?
As you can see we are getting closer but there are still a few issues:
- The asphalt is still too “shiny”
- The rails don’t look particularly shiny
- The asphalt, rails and groove in between rails all look the same, would be nice to have more color difference
Improving with three.js
Why doesn’t my output look the same as the example render? PBR implementation will always vary depending on platform or engine shader used to interpret the textures. “Comparing different rendering engines does not always work as expected since shaders are implemented differently. Even when using the same environment map, lighting can vary quite noticeably.” Test in your target platform and modify from there. Michael Herzog
renderer.toneMappingExposureto lower values than the default of
0.5. Michael Herzog
Improving with Photoshop
I manually edited the
diffuse texture maps to achieve my desired look.
To reduce the shinyness of the asphalt, I edited the
roughness texture map to significantly increase the brightness of the asphalt section. I also edited
metalness to ensure the proper appearance of the “tram grooved rail.” Finally, I slightly darkened the asphalt color and made the “trough” between the tramrails nearly full black with full roughness / no metalness to look like a recessed shadow in between the rails.
Here are the before / after texture maps:
I almost forgot the last step —remember to recombine these new roughness and metalness maps into a new combined AO/Roughness/Metalness texture using the Image Magick instructions above.
Just for fun here’s what the old and new look like:
Finally after incorporating the three.js improvements along with manual editing of the textures resulted in an output that looks much better! The one on the right is very close to my “artistic goal” with this material :)
View the final project in your browser: https://aframepbr.glitch.me/
Remix your own A-Frame PBR project: https://glitch.com/~aframepbr
More things to research and fix
- Would be nice to allow the
pbrcomponent to accept paths for texture files instead of hardcoding them in the component
- Investigate performance of PBR texture vs. simple image map, especially on Quest
- What does it actually look like in the Quest? Good but laggy, lower performance? Needs more digging.
- How do you extend this beyond current length without creating more draw calls? Will the uv2 trick still work? Time to use blender instead?