Plumbing for the Metaverse with Universal Scene Description (USD)
USD’s standardization for interchange is exactly what’s needed to stitch the open metaverse together
By Michael Kass, Senior Distinguished Engineer and Overall Omniverse Software Architect, NVIDIA
In 1992, science fiction writer Neal Stephenson coined the term “Metaverse” for a large-scale shared virtual world where people can interact with one another in a simulation of reality. The notion is certainly compelling. Why connect to each other in the 2D world of the current internet when we have the technology to interact in 3D?
Today, as virtual worlds pervade the world of entertainment and as a variety of businesses start to deploy “Digital Twins” to simulate and control real buildings, factories and infrastructure, it’s clear that the Metaverse no longer belongs in the realm of science fiction. It has entered the realm of the possible. But what will the full Metaverse look like? And how will it come into being?
Certainly, there will be many Metaverses. Some will be proprietary. Many of our present multi-player game environments already resemble Stephenson’s description to no small degree. But the value of a fully open Metaverse is too great for it not to come into existence. As we’ve already learned with the worldwide web, when everything freely connects to everything else, the network effects are immense.
Can our current web evolve into the open Metaverse? There have certainly been attempts in this direction. For decades, the computer graphics community has taken steps to integrate 3D into the World Wide Web. Early efforts included VRML and X3D. Today we have standards like WebGL, WebXR, glTF and the coming WebGPU. But none of these efforts has produced anything truly resembling a Metaverse.
At the core of any true 3D web or the open Metaverse, we need an open, powerful, flexible and efficient way to describe the shared virtual world. This will not be an extension to HTML or a javascript rendering library. And it will not be created by a standards committee. It will be an open source 3D scene description honed by years of use under challenging circumstances.
We believe the right description of shared virtual worlds for the open Metaverse already exists. It is Pixar’s open source USD: Universal Scene Description. USD evolved from Pixar’s need to represent film-quality virtual assets, scenes and animation in a way conducive to effective teamwork among artists and interchange among tools. Its features for teamwork are exactly what’s needed for the collaborative and social aspects of the metaverse. Its standardization for interchange is exactly what’s needed to stitch the open metaverse together.
USD is a scene description: a set of data structures and APIs to create, represent and modify virtual worlds. The representation is rich. It supports not only the basics of virtual worlds like geometry, cameras, lights and materials, but also a wide variety of relationships among them, including property inheritance, instancing and specialization. It includes features necessary for scaling to large data sets like lazy loading and efficient retrieval of time-sampled data. It is tremendously extensible, allowing users to customize data schemas, input and output formats and methods for finding assets. In short, it covers the very broad range of requirements that Pixar found necessary to make its feature films.
At its base, USD represents everything with a series of properties and values that are organized into higher-level schemas. For example, a mesh is encoded in a schema that brings together properties to describe the vertices of the mesh, their connectivity and any other related information. These properties and values are grouped into layers which serialize into separate files.
Layers are probably the single most innovative feature of USD. Conceptually, they have some similarities to layers in Photoshop: the final composite is the result of combining the effects of all the layers in order. But instead of modifying the pixels of an image like Photoshop layers, USD layers modify the properties of the composed scene. Most importantly, they provide a powerful mechanism for collaboration. Different people can modify the composed scene on different layers, and their edits will be non-destructive. The stronger layer will win out in the composition, but the data from the weaker layer remains accessible. Beyond direct collaboration, the ability that layers provide to non-destructively modify what others have done enables the kind of composability that has made the traditional web so successful.
The key to the functioning of an effective metaverse is replication. For a variety of people to share a virtual world, the appropriate parts of that world have to be replicated wherever the relevant rendering process lives. Of course, the web already has a variety of replication mechanisms. They include distributed databases, CDNs and caches of various kinds. But replicating a very complex 3D virtual world has its own special challenges. If the HTML for a web page changes, it’s possible to resend the whole modified HTML. For a virtual world described by hundreds of megabytes, this is simply not practical. So any viable open Metaverse must have the ability to replicate by sending incremental updates that specify only what changes.
As distributed by Pixar, USD lacks such a replication mechanism. But because of its extensibility and because everything in USD is described by properties and values, we at NVIDIA have been able to build an effective replication system on top of USD. The programmer can use the standard USD API to query a scene and change it. When desired, the programmer calls the usual “save” function, and our library constructs a difference between the original and the modified USD scene. That difference can be broadcast to the other connected participants. At the receiver, the difference is applied to bring the receiver’s world into synchronization with the senders’. This is the core of NVIDIA Omniverse.
Equipped with this incremental replication system, we believe that USD is the right way to build the open Metaverse. We have already connected a wide variety of popular content creation, simulation and other tools this way. We invite others to join us in doing the same. We are investing heavily in the overall USD ecosystem to build out all the remaining missing pieces.
At GTC 2021, we are hosting a panel entitled “The HTML of 3D Worlds — USD’s Next Evolution.” Register for free, then join our distinguished technical leaders as they talk about the potentials of USD for 3D creators in all industries. Panel participants include luminaries from CGTrader, Epic Games, Esri R&D Center Zurich, Kohn Pedersen Fox Associates, Pixar and Siemens.
If you’re new to USD and interested in learning more, you can take the Getting Started with USD for Collaborative 3D Workflows course or check out other Omniverse-related resources below.
Resources:
Meet the Author
Michael Kass is a senior distinguished engineer at NVIDIA and the overall software architect of NVIDIA Omniverse, NVIDIA’s platform and pipeline for collaborative 3D content creation based on USD. Michael is an ACM Fellow, received a Scientific and Technical Academy Award in 2005, the SIGGRAPH Computer Graphics Achievement Award in 2009, and was inducted into the SIGGRAPH Academy in 2018. He holds 29 issued U.S. patents and was honored in 2018 as Inventor of the Year by the NY Intellectual Property Law Association.
