Introducing Worldview
At Cruise, hundreds of software and test engineers interact with data collected on the road and in simulation by our autonomous vehicles. The AV Tools team builds applications to visualize, explore, and analyze this data. Our users need to understand what the car “sees” in its environment and what it’s planning to do — this includes point clouds with tens of thousands of LIDAR returns, hundreds of objects tracked by the AV, detailed street mapping data, and other custom visualizations built by Cruise engineers.
Like many other teams at Cruise, we build our tools for the browser, and React is a standard across all our projects. To reduce complexity in our apps and improve the developer experience while building these complex spatial visualizations, we created a React library that makes it easy to render 2D and 3D scenes. We call it Worldview.
WebGL is hard
GPU programming is powerful, but it can be overwhelming to learn. The browser presents a comprehensive API for compiling and linking shaders, uploading vertex array buffers and textures, and drawing objects — but it’s extremely verbose, complex, and hard to debug. As we iterated on our tools, we experimented with several frameworks built on top of WebGL.
We settled on regl, a library that hides WebGL’s complexity behind a simple API, translating “props” into low-level primitives and simplifying the execution of GPU draw calls — similarly to how React uses props and state updates to abstract away the manipulation of DOM elements.
regl provides a familiar React-like interface, while still giving us direct and complete access to WebGL features such as textures, framebuffers, and instancing. It allowed us to focus on shipping features instead of wading through WebGL syntax.
Cruise 💕 React
As other front-end engineering teams at Cruise began to use our regl-based renderer in their own projects, it became clear that some React design patterns were still difficult to reconcile with regl’s API. Despite the similar props-based paradigm, some GL expertise was required to create custom visualizations and integrate them effectively with our renderer.
To further ease the development process, we abstracted away the boilerplate code required to render common shapes and datatypes with regl by creating a common library of React components. This enabled other teams to reuse our work easily, while still allowing for customization.
This library became Worldview. It provides 2D and 3D cameras, mouse and keyboard movement controls, click interaction, and a suite of built-in drawing commands. Now our engineers can build custom visualizations easily, without having to learn complex graphics APIs or write wrappers to make them work with React.
Open-sourcing Worldview
While our initial development of Worldview was tightly coupled with Cruise’s proprietary visualization needs, the API boundaries became clearer as the library matured, and we separated Worldview into its own package, which we’re releasing as open source software under the Apache 2.0 license. We hope Worldview will lower the barrier to entry into the powerful world of WebGL, giving web developers a simple foundation and empowering them to build more complex visualizations.
What’s next?
We’ll continue to release more code as we further develop Worldview and other visualization libraries. We’re excited to receive feedback and contributions from the community — pull requests and issues are welcome. Enjoy!
Jacob Bandes-Storch, Brian Carlson, Jan Paul Posma, Audrey Li, Esther S. Weon
