Uber’s vis.gl brings glTF to geospatial data visualization
The following was originally posted to the Khronos® Group’s blog “Uber’s vis.gl brings glTF to geospatial data visualization”
Authored by Georgios Karnas, Ib Green, Travis Gorkin and Xintong Xia
In 2016, the Uber Visualization team released an open source version of deck.gl and luma.gl, two Khronos Group WebGL™-powered frameworks for visualizing and exploring huge geospatial data sets on maps. Since then, the technology has flourished into a full-fledged suite of over a dozen open source WebGL and GPGPU data visualization libraries and tools, known collectively as vis.gl. loaders.gl, the newest addition to the vis.gl family, adds support for loading and rendering glTF™ assets across the tech stack. This unlocks the ability to include rich 3D content within data visualization applications built using luma.gl and deck.gl, enabling a variety of interesting new use cases. In this post, we’ll show some applications and walk through how you can use deck.gl and glTF, Khronos’ open standard 3D file format, to quickly create a geospatial data visualization that renders tens of thousands of 3D models.
The Case for glTF
Prior to glTF, it would not have been practical to add support for high-quality 3D models to a visualization framework like deck.gl. Being able to mix 3D models into visualizations is, of course, appealing, but the work involved would have been too big and difficult to prioritize, as data visualizations are typically more focused on abstract visualization types like scatterplots, lines, and extruded geometric primitives.
However, glTF has quickly risen to become a major format for 3D assets in the WebGL world: glTF is supported by many major companies and products; online model marketplaces support downloading models in glTF; many popular tools support importing and exporting models in glTF, including the open-source Blender. Additionally, glTF models can now be loaded into a growing number of WebGL frameworks, as well as opened in the Windows Explorer and dropped right into in the Facebook feed. glTF is not only a format but an ecosystem, and we wanted to let deck.gl users leverage the benefits of that ecosystem.
A major upside of the wide adoption of glTF is that the same models can now be loaded into most major WebGL frameworks. Thanks to a well-defined, modern Physically Based Rendering (PBR) material and lighting model, glTF assets render identically (and look spectacular) regardless of where they are loaded.
By following the clear and concise glTF specification, building the Khronos reference PBR shader implementation, and consulting many invaluable open source resources, deck.gl was able to relatively quickly implement support for importing and rendering high-quality 3D models at a quality which is at parity with the best gaming focused WebGL frameworks.
To us, this is a testament to the power of glTF and a major example of the benefits of open, royalty-free, and thoughtfully-defined standards within the web and computer graphics industries.
glTF for Data Visualization
Earlier this year, the Uber Visualization team, in partnership with Uber’s Advanced Technology Group (ATG), released the open source AVS.auto system. AVS.auto leverages the vis.gl tech stack for visualizing and exploring autonomous vehicle data, such as point clouds from LIDAR sensors and predicted vehicle trajectories. AVS.auto and deck.gl are being adopted across a wide range of autonomous companies to build tools that render increasingly realistic scenes containing real-world objects, like vehicles, bikes, pedestrians, and other street/city features.
In our own tools, we use the visual difference between realistic vs. abstract visual elements as part of our design vocabulary. We use more realistic designs (such as glTF models) for real world objects and contrast that with abstract “Tron”-like graphics that show paths, trajectories, and point clouds to represent what the autonomous system is seeing, predicting, or planning to do. In addition, the geospatial features of deck.gl enable glTF models to be correctly positioned on top of base maps and visually interleaved with deck.gl’s standard abstract 3D visual elements.
Another interesting use case involves replacing abstract visual elements (such as points, lines, or arcs) in deck.gl visualizations with instances of glTF models that represent something concrete about the data.
deck.gl now offers a ScenegraphLayer that accepts a glTF scenegraph. Like all deck.gl layers, the ScenegraphLayer also accepts a table of data, and each row in the supplied data table is mapped to one instance of the glTF model during rendering, with offset, rotation, and scale extracted from each table row. The ScenegraphLayer uses instanced rendering on the GPU (the same technique that is used to performantly render large numbers of similar soldiers, vegetation elements, etc. in games), which enables remarkably performant rendering of thousands of identical models.
How to Use glTF in luma.gl, deck.gl and nebula.gl
It is very easy to create a glTF layer in deck.gl; all you have to do is import ScenegraphLayer and pass the proper parameters. Here is an example that allows you to place glTF Meshes in NYC: in our case, avocados. This could represent places where you can buy good produce or fruits.
This example leverages nebula.gl, another member of the vis.gl family that provides high performance, interactive editing of deck.gl layers. The result demonstrates a geospatial glTF editor example in less than a page of code:
You can move the map or rotate it using Option+Drag. Notice that you can have a 360-degree view of the loaded model as it rotates with the map!
The Future of glTF and vis.gl
Our frameworks have benefitted from the work done by many others on the glTF standard, and we want to give something back to the community. As a first step, we have open-sourced the framework-independent glTF and Draco loaders that we developed to support glTF in deck.gl.
These loaders became foundational building blocks for our new companion framework loaders.gl. This framework provides a growing suite of loaders for important 3D, geospatial, and visualization focused formats, including glTF and Draco.
The GLTFLoader in loaders.gl loads all the Khronos reference models and supports all incarnations of the glTF formats (binary GLB format, base-64 encoded JSON, and JSON with linked binary files) and also supports decoding of Draco compressed meshes. Support for additional extensions (like KHR_lights_punctual) is in progress.
Khronos, EGL, glTF, NNEF, OpenVG, OpenVX, OpenXR, SPIR, SPIR-V, SYCL, Vulkan and WebGL are trademarks or registered trademarks of The Khronos Group Inc. OpenCL is a trademark of Apple Inc. and OpenGL and OpenML are registered trademarks and the OpenGL ES and OpenGL SC logos are trademarks of Hewlett Packard Enterprise used under license by Khronos. All other product names, trademarks, and/or company names are used solely for identification and belong to their respective owners.
