Adventures in 3D model formats
At Goodboy studios we have been hard at work on our 3D engine in an effort to understand all facets of 3D and to figure out how we can turbo charge the performance of our web experiences and games. Recently we have been focusing on which runtime 3D formats we should support. There are many out there. Too many in fact! But which ones are best for the web?
Picking a format
We explored a few different 3D models when deciding what we would like our engine to use. Here’s our discoveries.
Off the shelf, What you should NOT use:
FBX — An industry standard, created for interop/transfer 3D data between apps but not good for runtime. Not only are these files much bigger they are also extremely complex to parse. This is because there are so many different ways in which an FBX model can be saved.
Collada — Whilst this has an easy to read XML format its HUGE and as with FBX, parsing all that XML is super slow. So not the best for runtime.
Off the shelf, what you should use:
GLTF — The best generic 3D model format for the Web is GLTF, it’s easy to read and the data is stored as binary so its inherently smaller than a text based format. This is a very well documented format so its super easy (compared to the other formats) to figure out how it works and parse it. Check out the ‘cheat’ sheet:
GLTF also supports a bunch of extensions, the best one being Draco compression can also be applied to GLTF making it even smaller. It crunches things down by a huge amount (up like 8–10 times smaller). It’s frankly insane how they managed to do that! Read about Draco here.
We make use of gltf-pipline, an amazing tool for working with gltf models in the command line. It makes it super easy to apply Draco compression and convert back and forth from glb and gltf formats (glb is like a single compiled binary format for gltf and its assets)
Definitely check it out if you are planning on working with the GLTF format.
Our 3D designers still prefer to give us FBX models as some of the software they use has not yet got a mature enough GLTF export for them to use. To mitigate this you can use this amazing FBX to GLTF converter:
We also created our own format — GBO:
One downside to Draco is that its decoder runtime is nearly 900 KB. When we think about small games (think tiny adver-games and that kind of thing) that’s a huge chunk of our file size budget. So we wanted to find a nice middle ground for our small models. Decent compression, but easy and small parser! Enter the GBO object!
A GBO object is a new simple file format that you can use instead of an OBJ one. The OBJ format is another very common one that is used extensively by many things (due to the fact It’s relatively simple to parse and is easy to use if you don’t need animation and just want mesh data). We made the GBO format as it drop in replacement that is faster to parse than OBJ’s and about 55–60% smaller. The key difference in our format are:
- The format is binary rather than text (so it's naturally smaller than the OBJ text format)
- We also used quantisation (basically converting all floats to uint16's) which essentially halves the data size (and the precision, but you won’t notice!).
- The data (once uncompressed) can be directly uploaded to the GPU, nothing else needs to be done.
You can check it out in action here. If you want check out the compressor the decoder and encoder can be found here:
We intentionally made it a generic format just in case anyone fancies using it. Here’s an example of how we load and parse a gb object. To convert an OBJ to a GBO using node:
import {convertObjToGbo} from 'gbo-reader';
convertObjToGbo('./myObj.obj', './myGbo.gbo');The best thing about the format is that parsing is extremely fast, waaay faster than an OBJ. The process of decoding this is actually super simple and fast too:
import {readGbo} from 'gbo-reader';
var oReq = new XMLHttpRequest();
oReq.open("GET", './myGbo.gbo', true);
oReq.responseType = "arraybuffer";
oReq.onload = oEvent => {
var arrayBuffer = oReq.response;
readGbo(arrayBuffer).then(o=>{
// o is a generic object..
// here we just use a pixi geometry
// but you can use your favourite 3D engine instead! var geometry = new PIXI.mesh.Geometry()
.addAttribute('uv', o.uv, 2 )
.addAttribute('position', o.position, 3 )
.addAttribute('normals', o.normals, 3 )
.addIndex(o.indices);
})
};
oReq.send()
Here the code for the decoder, as you can see the reason it’s fast is because there is not much code!
var readGbo = function(buffer)
{
return new Promise((resolve, reject)=>{
var out = {};
var index = 0
var stream = {buffer,
uint16:new Uint16Array(buffer),
float32:new Float32Array(buffer),
index};
// first read format
var format = {};
format.size = stream.uint16[index++];
format.indexSize = stream.uint16[index++];
format.bounds = {};
var bounds = format.bounds;
index = 1;
bounds.minX = stream.float32[index++]
bounds.maxX = stream.float32[index++]
bounds.minY = stream.float32[index++]
bounds.maxY = stream.float32[index++]
bounds.minZ = stream.float32[index++]
bounds.maxZ = stream.float32[index++]
bounds.sizeX = bounds.maxX - bounds.minX;
bounds.sizeY = bounds.maxY - bounds.minY;
bounds.sizeZ = bounds.maxZ - bounds.minZ;
// read the positions..
out.position = new Float32Array(format.size * 3);
out.uv = new Float32Array(format.size * 2);
out.normals = new Float32Array(format.size * 3);
out.indices = new Uint16Array(format.indexSize);
index = 14;
// unpack positions..
for (var i = 0; i < format.size * 3; i+=3)
{
var x = (( stream.uint16[index++] / 65535 ) * bounds.sizeX ) + bounds.minX;
var y = (( stream.uint16[index++] / 65535 ) * bounds.sizeY ) + bounds.minY;
var z = (( stream.uint16[index++] / 65535 ) * bounds.sizeZ ) + bounds.minZ;
out.position[i] = x;
out.position[i+1] = y;
out.position[i+2] = z;
}
// unpack uvs
for (var i = 0; i < format.size * 2; i+=2)
{
var u = stream.uint16[index++] / 65535;
var v = stream.uint16[index++] / 65535;
out.uv[i] = u;
out.uv[i+1] = v;
}
// unpack normals
for (var i = 0; i < format.size * 3; i++)
{
var n = ( ( stream.uint16[index++] / 65535 ) * 2) - 1;
out.normals[i] = n;
}
for (var i = 0; i < format.indexSize; i++)
{
out.indices[i] = stream.uint16[index++];
}
resolve(out);
})
}
module.exports = readGbo;Conclusion
There are so many different 3D model formats out there (and we just added one more!). The important thing to know is that most of them were not designed to be used on the web where file size and decoding performance are a priority. Luckily, the future here is led by the GLTF format flying the flag for the web! Whatever your engine of choice, here are the key takeaways:
- DO use GLTF as a runtime model format for the web.
- DO use Draco for compressing large models.
- DO try out the GBO format instead of OBJ if you are using small models with no animation.
- DON’T use Draco if file size is a concern (advergames, banners)
- DON’T use FBX or Collada as runtime formats because they are too big and too expensive to parse.
Hope you found this useful! Feel free to follow me on twitter to keep up to date with all things Goodboy tech! Oh, and don’t forget to check out our models in action in our fun Ziggy Piggy experiment here!
Thanks for reading!
