Instancing with three.js — Part 2
In this part we’re going to take an example that could potentially benefit from instancing and apply it!
Well, let’s find one, and analyze it. This one seems like it will do.
We would clone the repo, spin up a local server, and edit this file:
https://github.com/mrdoob/three.js/blob/dev/examples/webgl_interactive_cubes.html
First, let’s see what we’re dealing with:
console.log(renderer.info.render.calls) //add right above the render call
renderer.render( scene, camera );
These are draw calls that we’re logging. Notice two things — the number is much smaller than the number of entities created in the demo (2000) and it changes between render calls. They are randomly scattered in space, and as the camera moves through it, majority of them gets culled. Different frames cull different objects.
Since these are cubes, in best case scenario we process 24 vertices with each mesh when our vertex shader runs. Because of culling, we only run it on roughly a quarter of the whole set. This is good on it’s own, but again, because of how GPUs work, it may be much more expensive to issue the draw call, than to have the device crunch some math. Let’s bring this number to 1.
We’re going to focus on this area of the code:
var geometry = new THREE.BoxBufferGeometry( 20, 20, 20 );for ( var i = 0; i < 2000; i ++ ) {var object = new THREE.Mesh( geometry, new THREE.MeshLambertMaterial( { color: Math.random() * 0xffffff } ) );object.position.x = Math.random() * 800 - 400;
object.position.y = Math.random() * 800 - 400;
object.position.z = Math.random() * 800 - 400;object.rotation.x = Math.random() * 2 * Math.PI;
object.rotation.y = Math.random() * 2 * Math.PI;
object.rotation.z = Math.random() * 2 * Math.PI;object.scale.x = Math.random() + 0.5;
object.scale.y = Math.random() + 0.5;
object.scale.z = Math.random() + 0.5;scene.add( object );}
We declare a box geometry geometry. We don’t want to change this much, since we still want to keep just one instance of this data on the GPU.
2000 times, we create a new Mesh called object. These meshes all point to the same geometry (reusing it) but each have a unique material with a different color.
Next we place the object somewhere in space by setting the rotation,position,scale .
Finally the objects are directly added to the scene.
This last point is a bit tricky for the example, it would be better if we added the objects to another node thats not Scene so let’s just take a note of this.
First we’re going to create an instance of InstancedBufferGeometry and copy the geometry contents in it:
var geometry = new THREE.BoxBufferGeometry( 20, 20, 20 );var instancedGeometry = new THREE.InstancedBufferGeometry() //this is going to wrap both geometry and a bit of the scene graph//we have to copy the meat - geometry into this wrapper
Object.keys(geometry.attributes).forEach(attributeName=>{
instancedGeometry.attributes[attributeName] = geometry.attributes[attributeName]
})//along with the index
instancedGeometry.index = geometry.index
Next we move onto the scene graph part, starting with the number of nodes 2000:
instancedGeometry.maxInstancedCount = 2000We need to do some prep work before we dive into the loop. Since we can’t store the entire matrix, we’re going to store 4 x vec4 for the sake of simplicity.
const matArraySize = 2000 * 4
const matrixArray = [
new Float32Array(matArraySize),
new Float32Array(matArraySize),
new Float32Array(matArraySize),
new Float32Array(matArraySize),
]This is the bucket that’s going to hold the matrices that would otherwise be referenced as uniforms from individual nodes. We can wire these to the instancedGeometry as such:
for( let i = 0 ; i < matrixArray.length ; i ++ ){
instancedGeometry.addAttribute(
`aInstanceMatrix${i}`,
new THREE.InstancedBufferAttribute( matrixArray[i], 4 )
)
}This takes care of the position, scale, rotation that we will encounter in the loop. There is one missing though, the property of the Material. We make another attribute to account for this:
const instanceColorArray = new Uint8Array(2000*3)
instancedGeometry.addAttribute(
'aInstanceColor',
new THREE.InstancedBufferAttribute( instanceColorArray, 3, true )
)Let’s dive into the loop:
for ( var i = 0; i < 2000; i ++ ) {
var object = new THREE.Object3D() //we'll swap out the mesh with this//we should keep the color though
const color = new THREE.Color(Math.random() * 0xffffff)//we're going to piggy back of the scene graph and keep this as is
object.position.x = Math.random() * 800 - 400;
object.position.y = Math.random() * 800 - 400;
object.position.z = Math.random() * 800 - 400;object.rotation.x = Math.random() * 2 * Math.PI;
object.rotation.y = Math.random() * 2 * Math.PI;
object.rotation.z = Math.random() * 2 * Math.PI;object.scale.x = Math.random() + 0.5;
object.scale.y = Math.random() + 0.5;
object.scale.z = Math.random() + 0.5;scene.add( object );object.updateMatrixWorld() //we compute the matrix based on position, rotation and scale//now that we have the matrix computed we need to transfer it to the attribute
for ( let r = 0 ; r < 4 ; r ++ )
for ( let c = 0 ; c < 4 ; c ++ ){
matrixArray[r][i*4 + c] = object.matrixWorld.elements[r*4 + c]
}//same goes for color
const colorArray = color.toArray().map(c=>Math.floor(c*255))for( let c = 0 ; c < 3 ; c ++ )
instanceColorArray[i*3+c] = colorArray[c]
}
Now for the gnarly part. The GLSL that needs to be injected into the material in order to apply this:
instanceMaterial.onBeforeCompile = shader=>{shader.vertexShader = `attribute vec4 aInstanceMatrix0;
attribute vec4 aInstanceMatrix1;
attribute vec4 aInstanceMatrix2;
attribute vec4 aInstanceMatrix3;attribute vec3 aInstanceColor;${shader.vertexShader.replace(
'#include <begin_vertex>',
`
mat4 aInstanceMatrix = mat4(
aInstanceMatrix0,
aInstanceMatrix1,
aInstanceMatrix2,
aInstanceMatrix3
);vec3 transformed = (aInstanceMatrix * vec4( position , 1. )).xyz;
`
)}
}
The result should be something like this:
This is enough to position the elements, and reduce everything to a single draw call. We’re not using colors yet, and the lighting is actually off.
The lights still think that there is a single cube centered around the origin in model space, we need to also transform the normals in order for lighting to take effect.
The color is usually controlled through a uniform in the fragment shader, but since we cant load a uniform, and are instead working with an attribute, we have to read it in the vertex shader and pass it as a varying. We add another transformation to the shader (argh, curse ye onBeforeCompile):
shader.vertexShader = `varying vec3 vInstanceColor;${
shader.vertexShader.replace(
`#include <color_vertex>`,
`#include <color_vertex>
vInstanceColor = aInstanceColor;
`
)}
`
We should also read this in the fragment shader. Looking at the tamplate, we can swap out the uniform for the varying, with another transform:
shader.fragmentShader = `
varying vec3 vInstanceColor;
${
shader.fragmentShader.replace(
'vec4 diffuseColor = vec4( diffuse, opacity );',
'vec4 diffuseColor = vec4( vInstanceColor, opacity );'
)}
`This should hook up the colors. The lighting is still off, and the demo may have complained so i commented out a bunch of the picking stuff.
The lighting is actually very involved to get right. This is because normalMatrix is a bit specific in how it’s computed. Still, we can take a stab at it, by just rotating with the local instance matrix:
shader.vertexShader = shader.vertexShader.replace(
`#include <beginnormal_vertex>`,
`
mat4 _aInstanceMatrix = mat4(
aInstanceMatrix0,
aInstanceMatrix1,
aInstanceMatrix2,
aInstanceMatrix3
);
vec3 objectNormal = (_aInstanceMatrix * vec4( normal, 0. ) ).xyz;
`
)Notice we set the .w to 0, this is to distinguish the normal as a direction and not a point in space. This allows the matrix multiplication to just rotate the vector without translating it. There is also a race condition and if the matrix is defined in this chunk, it should be omitted from the begin_vertex but i just renamed it :)
This takes care of making the rendering portion work again. The nodes are scattered, colored correctly, but rendering at one draw call.
CPU side stuff
Let’s rewire the intersection functionality. This approach would probably work better with GPU picking, but for the sake of the example, we’re still going to do things on the scene graph and CPU.
var camera, scene, raycaster, renderer;var intersectsScene
We introduce a scene in parallel. We can instantiate it right before the loop:
intersectsScene = new THREE.Scene()for ( var i = 0; i < 2000; i ++ ) {
In the loop we revert back to the Mesh instead of Object3D and add it to the new scene. This is because we piggy backed off the Object3D and it’s convenience method to turn position and rotation into a Matrix4 . We could have just used the matrix without the object, but either way, now we need the actual geometry at some location in space to raycast against. We pass the reference to the same geometry, and wrap it in a scene graph node Mesh . Since the matrices get stored in attributes ordered, we keep the index so we can reference them later:
const object = new THREE.Mesh(geometry)object.userData.index = iintersectsScene.add( object )
We still let the logic populate the instancing attributes.
It would be wise to do something like this right after the loop. Since this scene is not rendered, the matrices never get computed:
intersectsScene.updateMatrixWorld(true)Now we move onto the logic and we tell the raycaster to use the dummy intersectsScene instead of the scene which we draw:
var intersects = raycaster.intersectObjects( intersectsScene.children );This should make the raycasting work, but the demo would complain when trying to set the color of an inappropriate material:
// if ( INTERSECTED ) INTERSECTED.material.emissive.setHex( INTERSECTED.currentHex );If we comment that out and log the hits, we will hit the meshes from the scene graph, even though we don’t draw them. We could log the INTERSECTED.userData.index and see which instanced mesh we’re hitting.
To explain this a bit - we have another scene graph, but it might as well be a branch of the main one, containing nodes Mesh. These have their position set for example, but do not render. The node wrapper takes Geometry which has a boundingSphere and places it in the position of the node. The ray then has something to hit. This is completely decoupled from the rendering logic which we do through the instanced geometry and a single Mesh.
There are a few lines left that manage which cube should be highlighted. We’re going to drop the currentHex name and setHex interface since we no longer have access to the interface that takes the hex value.
This logic uses the emissive property, which we need to add to the shader.
At the very top of the script, we’re going to add some globals for convenience:
var container, stats;
var camera, scene, raycaster, renderer;
var intersectsSceneconst red = [255,0,0] //a constant colorconst instanceEmissiveArray = new Uint8Array(2000*3) //the emissive arrayconst emissiveAttribute = new THREE.InstancedBufferAttribute( instanceEmissiveArray, 3, true )
emissiveAttribute.dynamic = true
We will also add a couple of functions:
function setEmissiveAtIndex( index, colorArray ) {
for( let i = 0 ; i < 3 ; i ++ ){
instanceEmissiveArray[index*3 + i] = colorArray[i]
}
}function getEmissiveAtIndex( index ) {
const res = []
for( let i = 0 ; i < 3 ; i ++ ){
res.push(instanceEmissiveArray[index*3 + i])
}
return res
}
This gives us some access to the pretty low level instance attribute. We provide the color (already typed as an array but could be THREE.Color ) and we provide the index.
We need to add this attribute to the instancedGeometry as well:
instancedGeometry.addAttribute(
'aInstanceEmissive',
emissiveAttribute
)Yet another unique transform is needed:
shader.fragmentShader = `
varying vec3 vInstanceEmissive;
${
shader.fragmentShader.replace(
'vec3 totalEmissiveRadiance = emissive;',
'vec3 totalEmissiveRadiance = vInstanceEmissive;'
)}
`Note that we need to modify some of the old ones, but the pattern is obvious. We need to read the attribute, save it to a varying and read from it.
Finally we replace the logic calls:
// if ( INTERSECTED ) INTERSECTED.material.emissive.setHex( INTERSECTED.currentHex );if ( INTERSECTED )
setEmissiveAtIndex(
INTERSECTED.userData.index,
INTERSECTED.currentColorArray
)
The getHex gets swapped out with our function:
INTERSECTED.currentColorArray = getEmissiveAtIndex(INTERSECTED.userData.index)At this point the demo should look more or less the same, and work as before.
If we up the number of cubes to 2¹⁶ we should still be at interactive rates (click here to see live). In fact this is nothing to render, the bottle neck may be the ray casts.
If we up the original demo to this number, it’s going to be pretty bad.
