AI, VP & TS — what they are
Some helpful definitions.
Artificial Intelligence (AI)
Let’s start with the hot topic that everyone is talking about at the moment thanks to the explosion of Chat-GPT and various other generative AI models. How does one begin to define Artificial Intelligence?! It seems that everyone’s definition is slightly different as AI isn’t just one thing. Perhaps it’s best to think of it as a constellation of many different technologies working together to enable machines to sense, comprehend, act, and learn with human-like levels of intelligence. It combines computer science and robust datasets to enable problem-solving. Technologies like Machine Learning (ML) and Natural Language Processing (NLP) are all part of the AI landscape, and each one is evolving along its own path.
I liked Gary Brotman’s provocation at Cambridge Tech Week:
Think of it as ASSISTED intelligence, not Artificial intelligence. It’s a tool to be used by humans.
Virtual Production (VP)
According to Dr Nina Willment, Research Association in Virtual Production at XR Stories, University of York:
In broad terms, virtual production is a way of making film and television which harnesses computer generated content that allows real-time visualisation and control of the digital environment in which you are shooting. Importantly, virtual environments are captured ‘in camera’ rather than added in post-production.
Similar to AI, VP is not a single technology, but more a collection of integrated tools and technologies that have matured to create an opportunity to dramatically improve on-set production processes. It draws on a range of visualising systems including Real-Time Game Engine Technology (such as Epic’s Unreal Engine or Unity Technologies’ Unity engine), Virtual and Augmented Reality systems, Motion Capture, Camera Tracking, Dynamic Lighting, LED Screens and In-Camera Visual Effects (ICVFX).
I had my first glimpse of how this work is made during the Creative Convergence training session, watching how a scene was shot inside the ‘Volume’ (a large wall of LED screens) at Target3D. It essentially empowered the filmmaker (in this case, Liminal’s Artistic Director Robyn Winfield-Smith), to create visually stunning environments that were highly realistic and — crucially — flexible. I witnessed how Unreal Engine generated these digital backgrounds and how she could then view and quickly manipulate them live on set. Actor Beth Cordingly sat on a real motorbike on the floor of the volume with LED screens behind her displaying a digital tunnel in real-time. Lighting could change in a second; day changed to night in an instance; the colour and speeds of passing vehicles could change in a beat. I saw first hand how VP can thus bridge the gap between the physical and digital worlds, allowing Robyn and her team to have real-time creative control over the digital elements. It is, without doubt, going to be an integral part of the future of film making.
One of the most revolutionary aspects of virtual production is that visual effects are applied to scenes during pre-production meaning that filming is more or less camera-ready by the time shooting begins. Indeed, thanks to virtual production, the old filmmakers’ mantra of “fix it in post” has rapidly evolved to “fix it in pre.” Adding and iterating on ICVFX elements during pre-production and production can facilitate tighter integration of live-action and computer-generated content while reducing post-production time. This ultimately saves both time and money, and reduces the risk of time and budget crunches at the end of the production.
Transmedia Storytelling (TS)
And finally, TS. Again, there are many definition out there, but to me it is the creation of dynamic story ecosystems where each new piece of media tells a different part of the narrative. It involves crafting intricate, multi-dimensional universes that are explored through diverse platforms. Each platform offers a piece of the overarching narrative, allowing audiences to delve deeper into the story through active engagement.
It is not a new concept; the study of TS — a concept introduced by Henry Jenkins, author of the seminal 2006 book Convergence Culture — is widespread across students, academics and practitioners. But its significance has heightened in the digital age that we find ourselves in today.
“Today’s ‘born-digital’ kids — the first generation to grow up with the Internet, born 1990 and later — crave gameplay in a way that older generations don’t. Most of them have had easy access to sophisticated games and virtual worlds their entire lives, and so they take high-intensity engagement and active participation for granted. They know what extreme, positive activation feels like, and when they’re not feeling it, they’re bored and frustrated.”
This quote from Jane McGonigal’s Reality is Broken: Why Games Make Us Better and How They Can Change the World while focusing on gaming, speaks volumes about the ongoing transformation in the media, arts and entertainment industries. The audience of today craves active participation, and traditional forms of storytelling simply don’t cut it anymore. This desire for engagement fuels the expansion of Transmedia Universes. It is now expected that these works provide a scaffolding for others to play, and that it is the creator’s job to ask the questions, but not presume the answers. This isn’t about just branching narratives anymore, this is about designing open worlds, a world which responds to the people who play within it.
