#TheUnrealLife — The Path to Pixel Perfect

Hex Mortis against our rear-projected, real-time background.

By: Shaunya Bhayani, RiTE Media

Imagine being immersed in any environment you could dream of with endless flexibility and possibilities, all harnessed in the power of a video game engine. Ever wish you could transcend into the medieval forests, alien moonscapes, and electric utopias that only occupy the furthest reaches of your mind? Yeah, me too! Well, no more wishing, it’s here and it’s rapidly evolving — a renaissance of the 21st century.

Market Challenges

It’s no secret that the film production process from conception through post is costly and time-consuming. Amongst movie studios, there lies a deep-rooted dilemma. Gone are the days of our local neighborhood Blockbuster stores. With the massive decline of theater and DVD sales, came the ascension of premium streaming services. Inevitably, major studios pulled back on small-budget films, giving precedence to films that are tentpole capable. The reality is that major studios now exist to build franchise brands. This carving out of mid-budget films is inherently limiting for independent studios, leaving movies that matter in the dust. As major studios hedge their bets using the top-down franchise approach, we, as independent filmmakers, don’t get to participate, unless we innovate. This is where Unreal comes in.

Our Narrative Test Case — Hex Mortis finds himself in a desolate, destroyed planet in the far-out universe. Why was he left behind and what does the future hold? Of course, this relates to his world, but it also relates to the present evolution of filmmaking. With this narrative test case, we’ve taken the first step on a long journey to reimagining how narrative experiences are brought to life.

What is Unreal?

Epic’s Unreal Engine has been home to some of the most successful games ever created, such as ‘Fortnite’ and ‘Pub G’; so powerful in fact, it’s no surprise Film & Television are transforming with it, as Virtual Production development and adoption continue to accelerate. Traveling to distant locations may prove challenging, due to global pandemics, prohibitive budgets, or zombie apocalypses. Virtual Production promises to unleash real-time photorealism, ultimately merging physical and virtual sets. When the entire production team can visualize the same world at once, silos are shattered.

Most major studios have already adopted some form of Virtual Production workflow. Thanks to recent advancements in GPU’s, gaming engines, micro-LED displays, and AR/VR technologies, intricate sets can now be fully virtualized before we ever shoot our first shot. The power of Virtual Production has been put to the test in high-budget tentpole series & films like Disney’s “The Mandalorian”; pushing this tech even further into the mainstream and enabling incredibly efficient productions.

The Mandalorian set against a real-time rendered environment.

Benefits of Virtual Production?

Virtual Production technology won’t be spelling out a eulogy for film professionals anytime soon — or more likely, ever. Fear not, this is actually a return to authenticity. Think about it… What a refreshing relief from the emptiness of green screens! This process invites a visceral connection between the director, talent, and environment; all on the Virtual Production stage. The phrase itself embodies the marriage of computer-aided production and visualization tactics and techniques, which are constantly evolving to be more accessible — meaning you don’t need massive budgets to conceive, compose, and captivate pixel-perfect scenes.

Where to Begin?

If you’re a curious person, you’re probably like, “alright I get the hype, but how do I learn this wizardry?” Serendipitously, the learning curve for Unreal isn’t insurmountable. If you’re coming from a VFX background or if you’re a creative coder, it’s not dramatically different. Unreal makes learning its engine accessible to everyone. Check out The Virtual Production Field Guide by Noah Kadner — it’s comprehensive and incredibly informative.

Unreal Engine 5

Epic Games broke the internet when they unveiled an early iteration of Unreal Engine 5. UE5 continues its mission to imbue creators across industries with the freedom and control to deliver cutting-edge content. And, with the application of tools like photogrammetry, we’re able to virtualize real-life physical assets giving it dynamic physics and lifelike animation. Additional refinements in the UI/UX will help the engine to become even more appealing to storytellers who aren’t too savvy in coding and video game development.

Part 2 of our Narrative Test Case — “Nailed It.” Hex Mortis reaches out as he searches for answers to escape from where he is stranded.

RiTE’s Quest

RiTE Media began a quest, following our intuition, beginning to R&D years ago as the real-time revolution first evolved the live events space. Back in 2016, we recognized the power of Virtual Production and how it would leave its imprint on content creation — focusing passion and resources towards our pursuit of the unseen. In June 2020, we released the latest of several narrative test cases we’ll be crafting through this medium. And, this is only the beginning. The next wave in storytelling will be the triumph of Transmedia experiences; the technique of telling a single story or story experience across multiple platforms and formats using current digital technologies. While we’re creating the movie, we’ll be creating the video game in concert.

Putting Things in Motion

RiTE Media is taking Virtual Production even further, successfully integrating our precision motion control robotics with real-time engines. This allows us to dial in environments, lighting, rigging, etc. while isolating and replicating precise variables. We can effectively emulate any dynamic environment in an incredibly small footprint. To augment the fierce combination that is Motion Control and Unreal, we coded and designed a LiveLink plugin in partnership with the Unreal team that allows the engine to receive critical position, zoom, and focus data from any robot, in real-time. This tethers our exact robotic movement to the virtual camera inside Unreal, which is what allows us to merge our [2] worlds with incredible precision and control.

Here’s an inside look at the bright minds that came together to make this happen. We’re proud to introduce our latest R&D breakthrough in the pursuit of the unseen: the world’s first integration of motion control robotics with real-time virtual production. For this test case, our greatest challenge was to create an expansive desolate environment in very limited space — using the unreal engine, rear projection, LED walls, a giant robot, popsicle sticks, and glue. All of this to capture @hexmortis at his lowest moment 🦾

Virtual production will become a symbiosis of human and technology, complementing each other’s strengths and connecting on a deeper level to capture visceral moments. New, higher-value, more fulfilling human roles will also emerge. Those who stay ahead of the game and continue to learn, experiment, and adapt to emerging technologies will do just fine.

And we intend to stay locked into this forever marriage between man and machine. Till death do us part.

Thank you to our partners — Sony, Marc Roberts Motion Control, and the Unreal Team. Without them, this journey would not be possible.