Experimenting with PBR textures using A-Frame and Three.js

Context

I’ve been working on an A-Frame hobby project Streetmix3D which creates a 3D version of streetscapes designed using the drag and drop builder Streetmix.net. (Streetmix started as a Code for America hackathon project and has been faithfully maintained as an open source project for the past seven years!)

The visual style of Streetmix3D started out with voxel stylized graphics created in Magicavoxel (left). I then experimented with using a photorealistic “custom photo” style texture I made using photogrammetry techniques and made into a simple repeating texture using Substance Bitmap-to-Material (right).

While I liked the look of the “custom photos style” (right) up-close, the “game ready” texture (center) looked better from a distance. I eventually found a reasonable quality output using a “Game Ready” style texture from Textures.com.

(See more angles and original images from this tweet)

While these texture improvements are more details than voxels, they are simply repeating “image map” textures from a single JPEG or PNG. They do not exhibit real life lighting effects aside from built-in defaults of A-Frame and three.js.

Moving beyond 80s and 90s style graphics on the web

The evolution of Streetmix3D textures reminds me of the evolution from polygon 3D style games from the early 90’s to real time texture mapping in mid to late 90’s. When I was a kid I played a lot of racing games including the PC MS-DOS game Stunts (solid 3d polygons) and later the Need for Speed series (image map textures and sprites).

Screencaps from Stunts 1990 (top) and The Need for Speed 1994 (bottom) PC Games (Stunts pics and download here NFS pictures and download from here, Stunts)

Now it’s time to try to take Streetmix3D to the next level out of the 90’s and into the new millennia of 3D graphics using Physically Based Rendering (PBR).

PBR with A-Frame and three.js

To learn about physically based rendering (PBR) and A-Frame I decided to start with a simple Project Goal: Display a plane in A-Frame that uses a PBR texture purchased from a third-party library.

Ingredients

I saw the best results using PBR with A-Frame and three.js with each of the following texture types to make the material look the best:

• Metalness
• Roughness
• Ambient Occlusion
• Image Map (aka Diffuse Map, Base Map)
• Normal Map

The scene will still render OK without all of these textures to varying degrees of success, depending on which are missing and what are your scene’s aesthetic goals.

Finding a PBR Texture from CGAxis

Buy your own copy of the texture from CGAxis if you want to use it in your project.

I was particularly interested in displaying a more realistic portrayal of the interplay between rail and asphalt for the tram and light rail lanes.

I found a beautiful looking “Road with Tram Rails” PBR Texture from CGAxis — it was only $5 and had everything I needed!* (*The metalness texture was missing at first but after a chat with support they quickly provided one.)

What’s inside of a PBR texture pack?

This is an example of the different files in a PBR texture pack as downloaded from CGAxis.

You may see varying files depending on the target platform and engine (Unity and Unreal seem to be the most common target platforms). Here are the files I found from the CG Axis “road with tram rails”:

• AO (ambient occlusion)
• Diffuse (aka base or image map)
• Glossiness
• Height
• Metalness
• Normal
• Reflection (aka specular)
• Roughness

To start making sense of these different files and their function I found this guide “PBR Guide Part 2” from Substance to be crucial helping me to understand the key parts of a basic PBR workflow. Highly recommended Sunday morning coffee reading!

Preparing textures for three.js

Discard — A-Frame is based on three.js which uses a “Metallic-Roughness” workflow and some of the files in the texture pack only apply to “Specular-Glossiness” workflow, so we can discard Glossiness and Reflection for this example.

Combine — Some of the texture images have only a single color channel (grayscale). These can be cleverly be combined into one image file by mapping each of the 3 grayscale images on separate channels for Red, Green and Blue. In fact we must combine these into one file for three.js.

Three.js expects Ambient Occlusion, Roughness and Metalness combined into one image file as RGB channels. You can combine this manually with Photoshop or a specialty tool like Substance Designer but that takes effort and those tools cost money, so I opted for a free command line solution instead!

Here’s a thumbnail of the texture image resulting from combining AO, Roughness and Metallic (ARM) into RGB color channels to prepare for three.js PBR

For this I used ImageMagick — a versatile command line image manipulation utility. Install it using your method of choice, I used the homebrew command line utility for MacOS to install:brew install imagemagick.

This installs a utility convert that you can use to combine files together:

magick convert texture_ao.jpg texture_roughness.jpg texture_metalness.jpg -channel RGB -combine texture_combined.jpg

If one of the textures is missing, you can substitute it with black using this method but it will affect the quality of your final output:

convert r.jpg g.jpg -background black -channel RG -combine combined.jpg

Read more about convert from the helpful ImageMagick documentation.

Displaying it with A-Frame

I could not find a way to define all these elements of PBR textures in A-Frame “out of the box.” I did find a great example implementation in the form of an answer to a Stack Overflow question How to Apply Environment Map to glTF in A-Frame written by Piotr Adam Milewski. He also provided a Glitch project proof of concept that renders a “Damaged Helmet” test object with an equirectangular environment map: https://stack-59625510.glitch.me/

A good starting point to build from is this glitch repo featuring the “Damaged Helmet” glTF model

I “remixed” Piotr’s glitch and replaced the glTF helmet with a plane geometry instead. This is practical for Streetmix3D roadway and in a more general sense to allow quickly experimenting with new textures before needing modeling software like Blender to apply it to a model.

I have renamed Piotr’s component to envmap and created a new component pbr that applies the specific textures to the plane material to support PBR. The syntax of the two components together are:

<a-plane pbr envmap></a-plane>

The pbr component does the following:

• Load the textures from hardcoded URLs using THREE.TextureLoader()
• Traverses through all the materials in the three.js object3d
• For each material: set the material’s maps to the loaded textures:

el.material.map = textureDiffuse;
el.material.aoMap = textureCombined;
el.material.metalnessMap = textureCombined; // can't set w/ A-Frame
el.material.roughnessMap = textureCombined; // can't set w/ A-Frame
el.material.normalMap = textureNormal;

Note that we must also “create a second set of UVs” for ambient occlusion to work properly in three.js. I found a few lines of JavaScript to do this (from somewhere in the three.js discourse group) and put them in the pbr component:

var geometry = mesh.geometry;
geometry.addAttribute( 'uv2', new THREE.BufferAttribute( geometry.attributes.uv.array, 2 ) );

What does it look like?

Viewing as a plane

Viewing as a sphere (left) to compare against rendering provided by texture library (right)

As you can see we are getting closer but there are still a few issues:

• The asphalt is still too “shiny”
• The rails don’t look particularly shiny
• The asphalt, rails and groove in between rails all look the same, would be nice to have more color difference

Improving with three.js

I posted for help in the three.js discourse forum and received the following tips to improve the scene:

Why doesn’t my output look the same as the example render? PBR implementation will always vary depending on platform or engine shader used to interpret the textures. “Comparing different rendering engines does not always work as expected since shaders are implemented differently. Even when using the same environment map, lighting can vary quite noticeably.” Test in your target platform and modify from there. Michael Herzog

If using ACESFilmicToneMapping then set renderer.toneMappingExposure to lower values than the default of 1 such as 0.75 or 0.5. Michael Herzog

Set material.metalness = 1, otherwise the metalness map will have no effect — this is why the rails didn’t look particularly shiny or reflective. Don McCurdy

Improving with Photoshop

I manually edited the roughness, metalness, and diffuse texture maps to achieve my desired look.

To reduce the shinyness of the asphalt, I edited the roughness texture map to significantly increase the brightness of the asphalt section. I also edited roughness and metalness to ensure the proper appearance of the “tram grooved rail.” Finally, I slightly darkened the asphalt color and made the “trough” between the tramrails nearly full black with full roughness / no metalness to look like a recessed shadow in between the rails.

Here are the before / after texture maps:

New asphalt is “rougher” (brighter white)

New metalness map has the correct part of the tram grooved rail highlighted

I almost forgot the last step —remember to recombine these new roughness and metalness maps into a new combined AO/Roughness/Metalness texture using the Image Magick instructions above.

Just for fun here’s what the old and new look like:

Finally after incorporating the three.js improvements along with manual editing of the textures resulted in an output that looks much better! The one on the right is very close to my “artistic goal” with this material :)

Old version (left), new final version (right)

View the final project in your browser: https://aframepbr.glitch.me/

Remix your own A-Frame PBR project: https://glitch.com/~aframepbr

More things to research and fix

• Would be nice to allow the pbr component to accept paths for texture files instead of hardcoding them in the component init function.
• Investigate performance of PBR texture vs. simple image map, especially on Quest
• What does it actually look like in the Quest? Good but laggy, lower performance? Needs more digging.
• How do you extend this beyond current length without creating more draw calls? Will the uv2 trick still work? Time to use blender instead?