Introducing Worldview

Image for post
Image for post

At Cruise, hundreds of software and test engineers interact with data collected on the road and in simulation by our autonomous vehicles. The AV Tools team builds applications to visualize, explore, and analyze this data. Our users need to understand what the car “sees” in its environment and what it’s planning to do — this includes point clouds with tens of thousands of LIDAR returns, hundreds of objects tracked by the AV, detailed street mapping data, and other custom visualizations built by Cruise engineers.

Like many other teams at Cruise, we build our tools for the browser, and React is a standard across all our projects. To reduce complexity in our apps and improve the developer experience while building these complex spatial visualizations, we created a React library that makes it easy to render 2D and 3D scenes. We call it Worldview.

Image for post
Image for post

WebGL is hard

GPU programming is powerful, but it can be overwhelming to learn. The browser presents a comprehensive API for compiling and linking shaders, uploading vertex array buffers and textures, and drawing objects — but it’s extremely verbose, complex, and hard to debug. As we iterated on our tools, we experimented with several frameworks built on top of WebGL.

We settled on regl, a library that hides WebGL’s complexity behind a simple API, translating “props” into low-level primitives and simplifying the execution of GPU draw calls — similarly to how React uses props and state updates to abstract away the manipulation of DOM elements.

regl provides a familiar React-like interface, while still giving us direct and complete access to WebGL features such as textures, framebuffers, and instancing. It allowed us to focus on shipping features instead of wading through WebGL syntax.

Cruise 💕 React

As other front-end engineering teams at Cruise began to use our regl-based renderer in their own projects, it became clear that some React design patterns were still difficult to reconcile with regl’s API. Despite the similar props-based paradigm, some GL expertise was required to create custom visualizations and integrate them effectively with our renderer.

To further ease the development process, we abstracted away the boilerplate code required to render common shapes and datatypes with regl by creating a common library of React components. This enabled other teams to reuse our work easily, while still allowing for customization.

This library became Worldview. It provides 2D and 3D cameras, mouse and keyboard movement controls, click interaction, and a suite of built-in drawing commands. Now our engineers can build custom visualizations easily, without having to learn complex graphics APIs or write wrappers to make them work with React.

Open-sourcing Worldview

While our initial development of Worldview was tightly coupled with Cruise’s proprietary visualization needs, the API boundaries became clearer as the library matured, and we separated Worldview into its own package, which we’re releasing as open source software under the Apache 2.0 license. We hope Worldview will lower the barrier to entry into the powerful world of WebGL, giving web developers a simple foundation and empowering them to build more complex visualizations.

Image for post
Image for post

What’s next?

We’ll continue to release more code as we further develop Worldview and other visualization libraries. We’re excited to receive feedback and contributions from the community — pull requests and issues are welcome. Enjoy!

Jacob Bandes-Storch, Brian Carlson, Jan Paul Posma, Audrey Li, Esther S. Weon

Image for post
Image for post

Cruise

A self-driving service designed for the cities we love.

Thanks to Jan Paul Posma, Esther S. Weon, and Adrian Macneil

Jacob Bandes-Storch

Written by

Rearranging bits for great good. Easily distracted by math, science, classical music, photography, &c. Working @Cruise. Formerly @Upthere, @HarveyMudd ’14.

Cruise

Cruise

Cruise is building the world’s most advanced self-driving vehicles to safely connect people with the places, things and experiences they care about. Join us in solving the engineering challenge of a generation: https://getcruise.com/careers

Jacob Bandes-Storch

Written by

Rearranging bits for great good. Easily distracted by math, science, classical music, photography, &c. Working @Cruise. Formerly @Upthere, @HarveyMudd ’14.

Cruise

Cruise

Cruise is building the world’s most advanced self-driving vehicles to safely connect people with the places, things and experiences they care about. Join us in solving the engineering challenge of a generation: https://getcruise.com/careers

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store