VR represents the abolishment of the framed and directed image. The spectator’s natural freedom of gaze puts her/him at the center of the work. This role denies passivity but is generously rewarded with an unmatched sense of presence in a virtual world.
I’ve recently created an artistic VR installation (The Lacuna Shifts) which gave me the opportunity to explore the potential of VR from an artist’s point of view. I’ve tried to understand the core qualities of this medium, as well as its strength and challenges, in order to use it effectively.
This is a short collection of my thoughts:
In various discussions it quickly became obvious that the term “VR” means different things to different people, often including AR, VR, XR, HMD, 360° Video. For this reason I’d like to define the kind of VR I’ll be focusing on.
- Immersive VR
where “VR” stands for a computer-generated, virtual world with real-time computation and therefore excludes 360° video
where “immersiv” suggests that this world takes up the entire field of view of the spectator in a stereoscopic fashion (and excludes CAVE-like settings and mixed reality applications)
- using a HMD (Head Mounted Display)
- and spatial Audio
Since the 80s VR appeared in regular intervals, always failing (it seems) on hardware/quality/convenience challenges. The fictional and philosophical concept of VR is of course much older.
A fuzzy medium
VR is a compound medium where no part is new in itself. Screens, computers, realtime graphics, controllers, spatial audio, motion tracking, etc have been around in many other settings for quite some time.
It eventually leads to the question if VR is a distinguished medium at all, because it can only be defined as a combination of display(s) and interface, bound together by a (not too strong) convention.
We can distinguish two features that sets it apart from other media:
- The virtual is taking up the entire field of view of the spectator. It is total.
- The virtual can be observed with a (seemingly) natural freedom of gaze, by moving head, body and eyes.
At its technical core lies the convention that maps the head movement in a direct fashion to a virtual camera in a computer-generated world. This calls for a 3rd item:
3. The gaze is present. The world knows exactly where the spectator is looking. In fact, the user’s gaze is constructing projections of this reality and is therefore part of it. This makes the HMD not only a display medium but also an interface.
We need to be clear that HMD is not only a display medium, it is also an interface. Interaction is not a choice, its unavoidable.
Although those specifics seem fairly simple their implication for a natural reception are huge allowing for an unmatched potential of immersion and presence leading to a low threshold for a “suspension of disbelief” even without the use of photorealism.
We further have to be clear about examining VR’s intrinsic qualities. This is obscured by the fact that as a medium of total simulation it has the potential to virtualize all former media and absorb their rules.
You could for instance reconstruct a living room and watch a movie on a simulated screen or visit a gallery and look at paintings. So we need to separate those mimetic features and look at its core.
The Age of Spectacle
VR is in a phase of spectacle and mimesis/skeumorphism. A “new” VR-audience is currently overwhelmed by the realism of this new medium alone. This calls upon the cinematic myth of Lumière’s “L’Arrivée d’un train à La Ciotat” (1895) screening where the viewers are said to have run away from the projection in awe of the realistic approach of a train.
The current VR-audience is — fueled by media-hypes — innocently demanding this wow-factor. And VR is certainly able to deliver. On the other hand it is easy to overlook the immense poetic capabilities of the medium that lies beyond the mere spectacle.
Comparing VR to established media
Indeed, VR is frequently being discussed in relation to cinema, especially when talking about narration. I find this very problematic because cinema is based on a sequenced directorial framing, which is not “naturally” available in VR.
In many VR critiques I detect a certain protective tendency toward the “old” media which is made possible through direct comparison. It almost feels like an anxiety of being rendered obsolete. This is of course not the case. VR won’t replace cinema. If anything it will free it from certain duties, that it wasn’t made for in the first place — like photography did for painting.
In many aspects VR is closer to theatre and performance than cinema (realtime, fourth wall, interaction, presence). But eventually this comparison falls short as well.
VR is primarily a medium for experiences not traditional storytelling — experiences invoke their own stories.
It is clear that VR is ready for emancipation: It needs to be seen as its own artistic medium, adding to the canon of film, theatre, architecture, literature,…
VR is still missing its own cohesive avantgarde movements which are required to find its place and establish its genres.
VR’s Specific Qualities
The experience of presence (sense of “being there”) with HMDs beats all other media
- Spatial Quality
An overwhelming part of a VR experience is architectural and relates to the perception of space which is strinkingly close to its physical counterpart. The potential of exploring the poetics of space beyond a simulation of the real, unlimited by laws of physics, is huge.
Overcoming the necessary realistic period is one of the most fascinating prospects of the medium, leading into a speculative perception of space and architecture that may border on drug induced hallucinations.
- Uncontaminated Context
John Berger in “Ways of Seeing” notices that in early days a painting was inseparable from the building it was produced for (eg church, palace,…) and formed a part of the inner memory of a building. With reproductions it entered our homes and contexts and therefore it’s context became anarchic.
VR through its almost total isolation (significantly more than cinema or whitespace galleries) offers the purest clean slate achievable today — no second screens (mobiles), no distraction. The artist has absolute control of what the audience sees, she/he can access an undivided attention which is unprecedented!
- The Gaze is present
Unlike in the real world the viewers gaze is general “knowledge” in VR and can be used constructively (magical activation of objects, leaving traces, becoming “manifest”).
Most of the following challenges are not to be seen as weaknesses. They carry a huge potential for artistic exploration and can be turned into strenghts. Only by neglecting them one might end up working against the medium.
Every immersion into virtual reality leads to an unavoidable extraction from the consensus reality. It brings upon a sense of vulnerability that is anti-social. The HMD represents a literal blindfold and many users are anxious about their unavoidably awkward appearance to bystanders.
This isolation also takes away the chance of a truly shared experience, which is one of our basic social needs which often makes an experience more valuable.
- Identity Crisis
Who are you (in here) and how did you get there, where is your body, and do you really need one? Does the world recognize you?
Right now the “Swayze effect” is almost unavoidable through the lack of sufficiently physical presence in the virtual world.
- Missing Genre Conventions & a crumbling 4th wall
VR has no generally accepted genre conventions and rituals yet. It is unclear what the spectator is supposed to do, what is asked of her/him: stay still, look around, engage,…
What happens to the fourth wall convention we accepted long ago? Because the spectator is put at the center of the world and is free to move around this metaphor can’t really work anymore (an invisible, spherical wall that moves with the viewer seems very far-fetched).
In a general sense, since its oral tradition, narration is a retrospective, not a real-time technique and is therefore not very well suited for VR.
Storytelling is based on non-realtime chronologies (jumps in time and space,...), which feels unnatural in VR.
We could again rely on the help of our trusted media e.g. by placing a narrator-character or a virtual screen into the world. In this case the rules of the “simulated” or “virtualized” medium apply and not the intrinsic specifics of VR. This is of course legitimate but often feels like a clumsy narrative crutch neglecting the medium’s own potential.
- Losing the Frame
Framing is a powerful device of suggesting a part-of-a-whole sentiment which is partially inaccessible, the possibility to refer to something outside of the frame. In VR this is lost. “I am the camera”!
- Natural Interactivity
Apart from the freedom of gaze the interactive element in VR seems to be way behind in terms of immersion an naturalism. We can only use very primitive methods to engage with the world. We use controllers instead of our hands, tactility and haptic feedback is missing. Although VR suggests a limitless space, our movement is limited by the invisible physical world. At the moment this represents the biggest immersion breaker.
- Transition (site specificity ?)
The onboarding and setting of the scene starts before the viewer puts on the HMD. How can we weave the transition from real to virtual space into the overall narrative to tie it to an actual place?
- Physical Strain and technical deficiencies
Last but not least the HMDs right now are far from ideal. They are heavy, strain the eyes and the limited field-of-view is still nauseous. During extended use, the viewer becomes painfully aware of the clumsy contraption she/he is wearing. It is clear however, that this will improve a lot when it achieves mainstream adoption. Right now it needs to be considered as an unavoidable fact which might even be partly woven into the narrative.