Notch 0.9.23.162 — A Virtual Production Powerhouse
From full XR through AR and green screen, the new update to Notch is packed with updates, features and fixes for virtual production (as well as quality-of-life updates that greatly simplify import and optimisation of 3D scenes and assets).
For the past year, we’ve been working hard on the next major update to Notch: a completely redesigned user interface, a huge improvement to workflow and user experience and many new features to the Notch rendering engine as well. This is going to be big — we’re incredibly excited, and we can’t wait to show you what’s coming.
However, the COVID-19 pandemic happened (and is still happening) and has taken a toll on visual art, design and music/show production around the world. With venues and audiences on lockdown, many productions have shifted from the physical to the virtual. This has placed new demands on both Notch and our community.
We’ve worked closely with our users as they produce world-class virtual productions (such as the MTV Video Music Awards, Amazon’s “Inside: The Boys” and Billie Eilish’s “Where Do We Go?” live streaming concert), and we’ve also been trying things out for ourselves.
We’ve learnt a lot along the way. As a result, we want to bring forward a few improvements into 0.9.23 that we feel will benefit those working in virtual production today, who continue to inspire and delight audiences around the world.
Those improvements are available today with an update that’s downloadable from our website.
Download the update and see the full changelog here!
We’ve created a quick video that goes over some of the news in this release:
Front/backplate
In XR virtual production, scenes usually need to be split between the backplate that is keyed in for greenscreen or ‘projected’ for LED stages, and the front plate (that gets composited into the output feed with an alpha channel). This process requires two separate yet simultaneous renders; the same scene but split into completely different parts.
We’ve developed a new workflow that not only makes it easy to manage plates themselves but also to preview your work in Notch Builder and deliver to media servers using Render Layers, including moving elements between plates for easy customisation on-site.
Notch is already natively supported by all major media servers.
Optimisation for virtual production
This update includes a whole host of optimisation workflows which apply to all Notch use but are particularly useful for virtual production
Virtual production has created new challenges for content delivery: everything is real-time 3D, not video, and it has to run smoothly at full frame rate throughout a show, with the added challenge of the same set of creative demands placed upon known from live event video: demanding clients, short deadlines and on-set adjustments.
We’ve introduced some new tools to enable 3D scenes to be created to run more efficiently, and to optimise them before delivery, while still providing the full creative freedom and live editability our users have come to expect from a Notch project.
You can now flatten 3D scenes on import, instance duplicate meshes and cache materials to textures. A new Freeze Geometry node allows you to import and arrange tons of geometry, and then freeze it in place for substantial performance gains.
We’ve also added the ability to automatically convert the 3D scene’s axes/flips on import which is useful when working with premade assets from different kits (which often can’t agree on what’s up or down).
Procedurally generated materials created with the Generators inside of Notch can now be cached to textures via a single button press. And the new Freeze Geometry node allows 3D scenes to be optimised by combining multiple meshes and avoiding the processing of generated and deformed objects.
In testing with our internal scenes as well as with projects from our beta user community, we’ve observed large virtual production scenes to shave off up to 15 ms per frame of render time, meaning being able to bring a heavy scene running at 30 FPS into the much safer area of 60 FPS, including media server overhead. This highlights the importance of easy-to-use optimisation, freezing and baking tools that are non-destructive, which is exactly what this update to Notch brings.
We’ve updated our manual to include a full overview of virtual production techniques in Notch, as well as a separate section talking about optimisations specifically.
NVIDIA AR SDK
Last month we announced that we were among the first NVIDIA partners to integrate their new AR SDK, which is part of the NVIDIA Broadcast Engine. The first feature we’re rolling out is the GPU-accelerated, AI-powered Face Tracker. It’s been tested by our beta community since and is now included in today’s release.
“Thanks to the new NVIDIA AR Broadcast SDK and the Notch implementation of the GPU accelerated Face Tracker, we now have a workflow that shifts the heavy lifting to the GPU. We are currently migrating from a configuration of TouchDesigner and Python + OpenCV, and getting this feature in Notch has been a game-changer for us with its highly responsive and accurate, low-latency tracking.”
Anders Granström, CEO PIXELFIELD
These GPU-accelerated SDKs deliver AI-powered features that Notch users can take advantage of in a soon-to-be-released software update. The first features coming to Notch are Face Tracking, Face Landmark Tracking and Face Mesh. You can read more about this in this separate blog post, and you can download the updated sample pack which includes an example project for the new face tracker.
Chroma keyer updates
We’ve made significant improvements to our already best-in-class chroma keyer. A new Transparency Output Mode allows you to generate a solid mask for alpha plus new Antialias Transparency property and Temporal Noise Reduction properties, improving the generated alpha channel and smoothing out temporal noise.
Updated templates
We’ve gone over all of the templates that ship with Notch Builder and updated them all to use the latest best practices and new features now available. If you haven’t checked them out yet, click the “Templates” tab when you open Notch Builder; there are templates to quickly get you started with everything from path tracing, smoke simulation, sound-reactive projects and now also a new virtual production template.
Full changelog and download
In addition to what we’ve briefly covered here, there are also over a hundred fixes, added features and changes in this release, so we highly recommend you update today!
