NVIDIA RTX Broadcast Engine Brings AI Enhancements To Livestreams
What if you could Livestream or record using your existing webcam and have a full-on greenscreen effect without the need for an actual greenscreen? This is one of the hurdles that NVIDIA plans to leap over thanks to their new NVIDIA RTX Broadcast Engine along with OBS and others. There are webcams and software already that try to do just this but they are mostly a miss with bizarre artifacts and ultimately falling short, but what NVIDIA does differently is they are taking advantage of the Tensor Cores in their RTX lineup of graphics cards and are putting the AI accelerators to use in several different ways through their RTX Broadcast Engine.
The new RTX Broadcast Engine SDKs include:
RTX Greenscreen, to deliver real-time background removal of a webcam feed, so only your face and body show up on the livestream. The RTX Greenscreen AI model understands which part of an image is human and which is background, so gamers get the benefits of a greenscreen without needing to buy one.RTX AR, which can detect faces, track facial features such as eyes and mouth, and even model the surface of a face, enabling real-time augmented reality effects using a standard web camera. Developers can use it to create fun, engaging AR effects, such as overlaying 3D content on a face or allowing a person to control 3D characters with their face.RTX Style Filters, which use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image. With the press of a hotkey, you can style your video feed with your favorite painting or game art.
NVIDIA has partnered with OBS for integrating their RTX Greenscreen right into OBS and will be showing it off at TwitchCon for those in attendance who want to see how effective it is before it is rolled out to the general audience in the coming months.
In addition to the RTX Broadcast Engine, we’ve recently seen several of the top software teams roll out support for the new NVIDIA Video Codec SDK. Elgato has used it for their new 4K60 Pro MK2 capture card and Discord as well for game streaming right from their app. The other big integration is through Twitch themselves with their TwitchStudio which I just got access too and will be visiting that in the coming days.
It has been and continues to be interesting to see NVIDIA find new and useful ways to tap into the hardware features that the RTX cards bring to the table and will be interesting to check some of these features out for myself when they become available.
https://www.yitit.com/nvidia-rtx-broadcast-engine-brings-ai-enhancements-to-livestreams-33892.htm
