The Content Conundrum: Creating the known by means of the unknown
As XR creatives, we filter our thoughts through lenses; Gaming and Theater are just two. Experience colors our perspectives. That’s natural. And few of us understand narrative XR as fully distinct media that operate on unique principles. There hasn’t been enough content from which to build consensus around effective artistic techniques. So, in considering content, we should acknowledge that we’ve learned a lot, yet have far more to understand. But we must first understand the two unknowns: Not understanding the strengths of VR, AR and MR, as narrative media and art forms; and subsequently how best to tell stories so that we exploit those strengths.
The First Unknown: Understanding the strengths of narrative XR
We approach these three ways. The first is carryover. It’s bringing techniques and concepts from mature forms into XR. So, we commonly see XR narratives that have game-like reward systems. They’re gamified narratives. But carryover includes less obvious techniques. One comes from film editing. And is known as the Kuleshov effect.
The Kuleshov effect happens when you watch a shot sequence. You create a meaning-on-the-fly that explains why image A to followed image B. Yet stands apart from both. Kuleshov is a mental process. And can occur in 360° video and handheld AR. In Lev Kuleshov’s original experiment, he showed food, followed by an actor in close-up with a neutral expression. People thought the actor was hungry. The same face shown after a seductive (seductive by 1918 standards) shot of a woman was said to “show desire.” But was “sad” when it followed a shot of a child in a coffin. You can read more here.
Carryover can too easily result in predictable content. That’s good from a commercial perspective. And affords consistency across experiences. And it benefits gamified narratives because people understand the interplay between game and story elements. However, it offers little for building strict narratives.
A strict narrative can branch. And the branching can feel game-like. But that doesn’t make it a gamified narrative. The branching is really a variation on the MYOA novel (Make Your Own Adventure). The strict narrative lacks a reward system. Instead, it offers satisfaction, pleasure, enlightenment, etc., “rewards” we ascribe to traditional art. However, the strict narrative is neither more nor less art than a gamified narrative. They’re different forms of the same thing, like EDM and baroque are both music ― though some early music enthusiasts would argue otherwise.
Carryover does show us what tools work, and which don’t. And those we keep will likely become part of XR languages, which convey information in a manner akin to film language (e.g. pan, close-up, etc.). But maybe a studio has poorly implemented a good idea. Or, they simply made questionable choices. For example, narrative design that clashes with the underlying story model. This happens in the PSVR title Déraciné. Whatever the case, we learn where carryover has limitations. And look to solve problems another way.
The second approach is exploration. It asks, “Where can we go from here?” And involves putting an unexpected spin on known practices or concepts. Many art projects do this. As do a few studios. Artists are more likely to “go bold” because they’re free from preconceived notions about how things have been done. On the other hand, XR veterans feel free to develop new ideas because the old ones, the status quo, are effectively a sophisticated sandbox. Either way, exploration may get us close to the 1915 moment, if not past it. In The Evolution of Narrative XR, I explained the moment as a series of widely-acknowledged artistic breakthroughs. They’ll trigger paradigm shifts as consequential for XR creatives, though less dramatic, as Steve Jobs’ iPhone introduction was for the mobile industry. In other words, when breakthroughs happen, everyone will know it.
The downside to exploration is that it’s random. And can unintentionally skew us toward a narrow future. We’d benefit from interesting possibilities while not realizing what we’d missed. This is especially true for AI, which can accidently cause a carryover effect. Algorithms might “bake in” assumptions about immersive stories as currently understood, on top of the type AI bias that has already proven problematic. And this at a time before the 1915 moment. But we can moderate potential effects through the third approach, theory.
XR theories explain how XR media work, the way film theories explain cinema; psychologically and emotionally in a narrative context, and at a basic level as art. Consider them formalized explorations. Yet they’re based on simple principles, such as “conflict, alignment and proximity,” for Proximity theory. The concepts double as checks to ensure that new ideas fit with the old.
Theories attempt to get at the essence of reality. How they do it, how they express it, depends on the theory. But each helps us break down immersive story mechanics so that we know why, say, violating personal space in a VR experience ― something normally avoided ― is in fact a good way to heighten dramatic tension in certain situations. This example also shows the connection between theories, exploration and the second unknown, appropriate technique.
The Second Unknown: Techniques natural to each medium
Storytelling techniques should not be confused with best practices. The latter help you set up an experience that avoids pitfalls. While techniques relate to artistic expression, moment-by-moment throughout the narrative. They can evoke a desired emotion by directing gaze toward a sequence of objects or events, known as montage bias. Position angle bias is another standard technique.
Position angle bias is when you agree with, or are sympathetic to, a character whose gaze aligns closely with yours. The film equivalent is camera angle bias/camera perspective bias. We find it elsewhere. Studies have shown that, when a camera has looked head-on at the defendant, juries are more likely to believe a videotaped confession is the truth; regardless of whether the “truth” may have been coerced. A camera that shows interrogator and defendant in profile tends to neutralize the effect.
What it All Means
The two unknowns are history repeating itself. Filmmakers had to dump the idea that movies were photographed plays, photoplays, before another idea, film language, could make sense. Theater was to early filmmaking what Gaming and Cinema are to narrative XR. But unlike then, we won’t see a clean break. Too many gaming elements have proven useful. Likewise, for some cinematic techniques. The future landscape of narrative XR will not resemble the present. We’ll find ourselves forced to ask new questions. And as creatives we should welcome the challenges they’ll bring, if we’ve already understood the current unknowns.
What are your thoughts on content?
