The Evolution of Narrative XR or: It’s like 1911 all over again

In her novel Gubi Amaya (published 1850 -1865), Juana Manuela Gorriti creates a world where fact can masquerade as illusion. Her protagonist Emma, disguised as a man named Emmanuel, visits the childhood home she fled during political upheaval. Yet, for all the evocative prose, true sight and sound eludes us. This 1840s Argentina remains a movie in our heads.

But now we have the media collectively known as XR. They can do what Gorriti never could. They can put you inside the ruined castles that’s on the estate grounds. They can transform you into Emma. Or into the men who, not seeing through her disguise, seem equally blind to their own realities. Those examples only hint at the power of narrative XR. It’s not the ability to step into someone else’s skin. That’s just role-playing. It’s living their truth, which is the essence of the immersive experience. Hence the name of this publication, Immersion XR.

I had planned to add the subtitle “Journeys in AR Storytelling.” But this is neither a how-to project nor a diary about making an AR film. And though I tackle XR issues from an AR-first perspective, those I discuss often apply just as well to VR. Yet, Immersion XR is in fact a journey, of the mind. It delves into psychology; into art theories and traditions; and examines the interplay between each one and technology. Like a persistent two-year-old, it asks “why,” as it critiques, as it challenges conventional wisdom. But it won’t provoke for the sake of it. At all times it seeks to spark debate and critical thinking. And in its small way to help shape this emerging art form that, whether you call it Spatial Storytelling, Immersive Storytelling or Narrative XR, has untapped possibilities.

OK. Where are we? You’ve probably read that VR, MR and AR as storytelling media are where motion pictures were in the days of hand-cranked cameras. This is true. But that era spanned from the cinématographe, an 1890s camera-projector from which we get the word cinema, to that masterpiece of Montage Theory, The Battleship Potemkin.

Immersive stories already have more sophistication than the Lumière brothers’ The Arrival of a Train at La Ciotat Station. In it, a train pulls into a station, where commuters get on; simple yet astonishing for 1896. By 1909, slapstick two-reelers were a motion picture staple. In 2019, we don’t yet have their XR equivalent, though one can argue that gamified narratives might qualify due to their sheer number. Of course, how we measure progress is more complex than that.

You obviously experience The Elder Scrolls V: Skyrim VR differently from Orson Welles’ Citizen Kane. Yet, the Immersion XR view is that both lie on a continuum. The term “gamified narrative” may seem an oxymoron in this context. It’s not. It refers to proactive interaction models that originated in, or whose concepts reference, gaming.

The first decade of the 20th century saw innovations that artists developed into film language. But nothing has emerged in XR with the same impact as the film close-up, circa 1900. Does this mean we’re in an 1890s redux? Not quite.

In those early years, a cinematographer like Billy Bitzer could push the medium forward through creativity. And often without reliance on the latest technology. Bitzer introduced nighttime cinematography when he lit a scene with magnesium flares. In contrast, technology plays a bigger more complicated role in narrative XR. And that affects creative expression. For example, Google Cardboard affords few technology-based options. The nature and quality of the experience relies almost entirely on creativity. At the opposite end of the spectrum is the HMD experience that cannot exist without AI. And yet they both share one thing: The flexibility to create any type of story structure and even define what constitutes a story. Cinema on the other hand, has remained a linear storytelling medium, regardless of artistic and technical breakthroughs. Even the triptychs in Abel Gance’s 1927 Napoléon illustrate linear subplots within a pre-defined narrative. Any kind of simple equivalence between Narrative XR and Cinema simply doesn’t work.

You might think, “So what? Does it matter that we track narrative XR evolution?”

Well, it does. Here’s why: As creatives we may want our stories to entertain, to educate, or move people to action; but in all cases we want stories to touch, touch emotionally the people who experience them. The easier we can identify tools that enable that; and build knowledge around them; the more engaging, emotionally satisfying or effective our endeavors.

Immersive stories do engage. They entertain. But the narratives, sometimes called films ― whose terminology I address in Words Fail Me: On the need for terminology unique to Immersive Storytelling ― do not exploit XR media the way an over-the-shoulder shot does in films. XR doesn’t have widely used techniques that achieve the same psychological effect.

And while Cinema has had movements, Italian Neorealism being one, Narrative XR isn’t there yet. It hasn’t spawned its own New Wave. It lacks bodies of thought as comprehensive as Montage Theory. And it hasn’t given us a breakthrough experience that has us saying, “Aha! That’s how to build a story. That’s how to exploit this new medium!” For Cinema, the breakthrough came in 1915 with the infamous The Birth of a Nation.

It’s difficult to discuss The Birth of a Nation in purely cinematic terms because many of its innovations are bound to the film’s racist content. I suggest reading Roger’s Ebert’s 2002 essay, which examines both in socio-historical context.

The 1915 moment for Narrative XR will arrive in a slow wave. That’s because the different media (i.e. VR, MR and AR) develop independently. So, we’ll see groups of breakthrough narratives. They’ll effectively use techniques that exploit characteristics unique to each medium. The techniques may not be innovative. Their usage will be. And neither will depend on proprietary technologies.

The stories themselves will fall into two categories: Originals and adaptations. It’s one thing to tailor a VR experience around existing storytelling methods. It’s quite another to develop methods that recreate the richness of a Gabriel García Márquez novel. And, importantly, retain his distinctive voice throughout. For those reasons, I believe adaptations will lead to the bigger breakthroughs. 100 Years of Solitude, using ARCore? Why not?

So, I say, “Welcome to 1911.” The date acknowledges progress. But progress that’s fragmented. Narrative XR has taken baby steps away from the Cinema, Gaming, and Art worlds. We should help it along through imagination; through exploration, which is but questioning the possible through action. What can we learn from Dujardin’s 1880s novel, Les Lauriers sont coupés, famous for its stream-of-consciousness observational style? This is the question in an upcoming story. Or, could we create shorthand for a character’s psychological state from a concept like Kabuki’s mie? Immersion XR will address that as part of a broader discussion about psychological depth. It’s one leg in the journey of the mind mentioned above. So, don’t just enjoy the ride. Take us into parts unknown if your wish. As it’s often been said, “The journey is the destination.”