Create Your Own Metaverse Applications with C++ and Python
Omniverse Kit 104 delivers an expanded toolkit and easy-to-use templates for building metaverse applications and extensions
By: Damien Fagnou, VP of Software, NVIDIA Omniverse
Today, NVIDIA announced Omniverse Kit 104 with exciting updates that allow C++ and Python developers to more easily create, package, and publish metaverse applications. With an expanded toolkit, increased flexibility, and easy-to-use templates, developers of various experience levels can build metaverse applications and extensions to accelerate industry-specific workflows.
Omniverse Kit is the software development platform housing a continuously growing array of features and building blocks for building metaverse apps, extensions, and microservices. Omniverse Kit 104 comes with exciting updates to key developer tools and extensions including:
- Open access to C++ extension development with templates and examples, in addition to existing Python templates
- A new application template so you can easily create your own apps
- Viewport 2.0 with advanced and open workflows
- New navigation possibilities for user interfaces in Omni.UI.Menu
- The ability to encapsulate extension features in Actions
- A centralized API and UI to manage Hotkeys
- New and improved documentation
All of NVIDIA Omniverse’s core applications are based on Omniverse Kit, including Create, View, Isaac Sim, DRIVE Sim, and Farm. These applications are all built upon the software stack with wide-ranging shared extensions that are broadly available to you, such as Omniverse Create, which comprises more than 300 extensions. Even the Extension Manager is an extension! This means that when you build extensions, you’re building in the same way that we do when we build Omniverse features.
Omniverse Kit’s runtime powers Omniverse simulations and experiences with components like RTX, PhysX, OmniGraph, and USD. There are many extensions that run independently of the runtime and also many extensions that combine the UI and runtimes and power of the kernel to create accelerated extensions on applications.
Open access to C++ extensions with templates and examples
Kit 104 now supports a new set of extension templates for C++ developers to create extensions with C++ plugins. These templates in GitHub contain various example extensions to act as references for developing UI widgets, USD interactions, and more, removing the need to create these from scratch and speeding up your application development.
The extension templates demonstrate how to:
- Create a C++ plugin that will be loaded at startup (omni.example.cpp.hello_world)
- Create a C++ node for omni.graph (omni.example.cpp.omnigraph_node)
- Create a C++ widget for omni.ui that has a property and draws a simple rectangle (omni.example.cpp.ui_widget)
- Create a C++ plugin that can modify the current USD stage (omni.example.cpp.usd)
Additionally, many new Python templates have been made publicly available in GitHub, including samples that range from creating UI scenes to doing scattering like geometric creations and commands.
Easily create your own apps with the Omniverse Kit Application Template
You can now create your own custom applications in Omniverse Kit and develop large, sophisticated applications like Omniverse Create and Omniverse View. The new application template makes it easier than ever for you to create your own apps leveraging technologies from the Omniverse platform like RTX, PhysX, Nucleus, OmniGraph, and USD.
The template can be used as a starting point and comes with a very simple “hello world” example that demonstrates how simple it is to develop apps with Kit. It also includes two more advanced apps that show how you can use the many extensions available in Omniverse to build feature-rich applications.
Advanced and open workflows with Viewport 2.0
Viewport 2.0 was built with the Python developer in mind to enable more advanced and open workflows. Create 2022.3 will feature a second viewport leveraging Viewport 2.0’s features and functionality, enabling creators to interchange between even more points of view in their design environment.
Viewport has been redesigned from the ground up with new extensions including:
- Default Viewport implementation (omni.kit.viewport.window)
- Viewport Menu system(omni.ki.viewport.menubar)
- Modular Viewport Manipulator system (omni.kit.viewport.manipulator)
- Utility functions and legacy viewport compatibility (omni.kit.viewport.utility)
More options for flexible menus with Omni.ui.Menu
Omni.ui.Menu has been revamped in Kit 104 to enable new possibilities for navigation and user interfaces with new features that:
- Fully control the look of a ui.MenuItem (ui.MenuDelegate)
- Enable easy procedural menu creation (ui.Menu.build_fn)
- Enable radio menu behaviors easily (ui.MenuCollection)
- Re-implement ui.MenuItem in Python to be drawn by delegate
Invoke and execute actions
Kit 104 brings the ability to encapsulate extension features in easy-to-call Actions that are programmable and available in C++ and Python. The Action window houses all the available actions, which can be created, registered and executed from any extension.
A centralized API and UI for hotkeys
Kit 104 introduces a centralized API and UI to create and manage your Hotkeys. Hotkeys are enabled by Actions and can be:
- Created or registered from any extension
- Associated with Actions
- Overridden and Managed by the User
- Hotkey binding can be “Exclusive” to a Window
- Discovered easily from the Hotkeys Window
New and improved documentation
The NVIDIA Omniverse documentation has been updated with user friendly navigation so you can easily find the answers you need when building your own application. The following are some key enhancements made to the documentation:
- The HTML has been refreshed so the build system produces enhanced indexes for different classes, along with a core index spanning all documentation.
- A new documentation builder that allows you to easily build documentation by writing markdown files that can be used interactively in applications or exported into traditional HTML to be used in web browsers.
- The documentation also now includes resources on style with style attributes lists for each widget.
New extensions built with Omniverse Kit
Perhaps the best place to see what’s possible in Kit is in the latest versions of Omniverse applications. This year, Omniverse Create has launched over 300 extensions built in Kit.
- Action Graph, an extension of Omnigraph in Create, is an exciting new example of what’s possible with Kit. Action Graph allows you to create event-driven behaviors and logic inside scenes with visual programming. Kit is also enabling improved Omni.ui.Graph extensions with the new GraphEditor.Core extension, which makes it even easier to develop graph-based extensions or applications.
- A new framework in Omni.ui called Omni.ui.scene allows you to build interactable UI for widgets and manipulators directly inside the viewport or 3D environment. The extension allows users to create shapes in a 3D projected scene and manipulate them easily with a gesture system.
- Deep Search window is an extension built with Kit that enables AI-based natural language queries into Omniverse Nucleus’s asset database to retrieve images, objects, or other assets. From there, the user can drag and drop the asset into their scene.
NVIDIA also recently introduced a new generation of live, collaborative workflows with USD and Nucleus called Omniverse Live. Built on Kit, layers of synchronizations are faster than ever before, enabling multiple users to collaborate on modifications of scenes from multiple locations in real time. Live mode works with any Omniverse App or Connector and allows creators to work together in seamless, non-destructive workflows.
Developers leading the way
Many developers are leading the way building useful extensions and connectors in Omniverse Kit that optimize and improve 3D workflows. Once ready for widespread use, developer partners and community members can publish applications, connectors, and extensions in Omniverse Exchange with the new self-publishing portal.
Some of the more recently published extensions and connectors include:
- Reallusion iClone 8.1.0 live sync Connector for seamless interactions between Omniverse apps and iClone 8
- The OTOY OctaneRender hydra render delegate, which enables Omniverse users to use OctaneRender directly in the Omniverse Create or View viewport
- The Nextspace digital twin platform extension for normalizing data and geometry to drive the use of AI, analytics, and simulation
- SmartCow’s Omniverse extension for synthetic data generation of large datasets of license plates for license plate recognition AI
Recently, NVIDIA finished its first Omniverse developer contest which brought in a plethora of innovative extensions submitted by users across the Omniverse community using Omniverse Code and Kit. You can read our blog on the winners to draw inspiration on designing your own Omniverse extensions.
Visit the Omniverse Developer Resource Center and the USD page for additional resources, view the latest tutorials on Omniverse, and check out the forums for support. Join the Omniverse community, Discord server, and Twitch Channel to chat with the community, and subscribe to get the latest Omniverse news.
Follow NVIDIA Omniverse on Instagram, Twitter, YouTube and Medium for additional resources and inspiration.
Meet the Author
Damien Fagnou is vice president, of software at NVIDIA, working on NVIDIA Omniverse, where he brings together his expertise in software and visual effects (VFX) production to help build Omniverse Kit and Omniverse Code. Previously, Damien was the senior vice president of technology for Technicolor, where he oversaw technology and software strategy for the VFX group, as well as CTO for MPC Film, working on Oscar-winning films “The Jungle Book” and “1917.” During that time, MPC also created the VFX for the groundbreaking “Lion King” remake. With almost 20 years of experience in VFX, Damien looks forward to contributing to the future of graphics software at NVIDIA.
