Interreality 101: wtf it is and why VR designers should care.
Five principles for designing interactions between virtual worlds and conventional people
Literally, it means “between realities.”
Here, I use it as a term that describes a reality where interactions between parties in virtual reality (VR) and parties in conventional reality (CR) occur.
Virtual and conventional realities are often positioned as being at odds with each other. Using a VR headset creates an immersive but also isolating experience. You lose connection to those sharing your physical space, and even conversation with CR parties becomes difficult to hear or address. As the industry continues to speculate about the potential of VR, its CR surroundings are typically ignored, or even considered a technological constraint. My research lies where these two realities overlap.
An interreality of social VR has not been widely explored except for a few games. These games mostly integrate CR observers in digital space via a computer or mobile device. How can we create social interactions via VR that integrate the VR actor into physical company?
By inverting the ways we traditionally think about perspective, presence, and control in virtual environments, we can leverage the conventional surroundings within which VR is bound — and create an interreal space where both VR and CR parties are equally engaged.
Over the course of 2 months, I have immersed myself in a variety of VR experiences, some games of my own creation, social VR platforms, and also conducted a co-creation workshop where we investigated new ways to use the hardware “wrong” in attempts to uncover some fundamentals of interreality.
Five core themes emerged from these experiments: physicality, collaboration, asymmetry, control, and space. Here are my thoughts about how to implement them.
1: Be physical
Be it bodily contact or elements of performance, interreality presents a multitude of opportunities to engage our entire bodies. For example, think about how touch can function as both input and feedback. In some scenarios, physical cues may be the only information CR parties receive about the virtual world. To a VR party, touch may reveal an invisible CR obstacle. It’s also worth considering how physical objects can be used to solve problems in virtual space. For the participants, and perhaps for the benefit of the audience, interreality provides the opportunity to exaggerate movement to the fullest degree — in attempts to convey, communicate, or simply entertain.
2. It feels good to work together
And it feels good to be together. Interreality makes fertile ground for exploring new types of socialization and alternative modes of communication. As compared to increasingly popular social VR platforms (which are ironically difficult to use in tandem with conventional company), interreality allows for more opportunities to leverage shared space so that it enhances the shared experience. Most interreality interactions also involve some form of collaboration. Working together increases the willingness of people to try again and again until their goal is reached; “failing” or “losing” just turns into more motivation to practice and improve and adapt together. But this togetherness often ends up being more important than any improvements alone. After all, it still feels good to be together.
3. Create imbalances
Imbalanced interactions are those that create the need to work together in interreality. VR can be used as a tool to provide information than CR cannot. But it can also be used in order to take away information and control. Consider all the things you can’t see while wearing a headset, as well as what you can. Awareness of these asymmetrical access points to each reality is critical when designing interactions across them. Gaps in knowledge between parties turn the unknown into something to be discovered. Imbalanced sensory abilities also frequently allow for the incorporation of a mediator into the interaction. The imbalances however, must be balanced enough to engage both realities equally, in order to avoid effects like visual FOMO on the part of the CR party. A nice trick to avoid this is to have a motion trace or some other artifact generated by the performance for the participants to look at afterwards.
4. Adjust the controls
Controllers aren’t just worse hands. They are new tools that have their own properties which are being ignored — both as virtual and conventional objects. Distributing controls across realities is a handy way to start exploring this. Pay attention to what the virtual representations of the controllers can tell you about the bodies behind them. Does their orientation tell you where people are? Would you be able to guess who is holding one from the personality of its movement alone? It is also important not to forget what roles an observer can play — and to remember you don’t need digital control to control a digital experience. Lack of agency is typically believed to cause frustrating tension in VR. But taking away or inverting controls can deepen an interaction when it creates imbalances that participants can reconcile.
5. Consider space
Virtual spaces have no rules. Virtual objects, environments, and physics are often designed to mimic CR, but shouldn’t be expected to do so. The properties of space — conventional and virtual alike — provide unique opportunities for interreality. Think about how you could store virtual items in physical space, and conversely, where material things might be virtually stored. Play with the roles people play in the space they occupy. To establish a shared spatial understanding between CR and VR parties, it can be helpful to limit the virtual space you create to the physical space it inhabits. Have fun with the dissonance you can create between what people expect from virtual vs. conventional space. Are all objects in the same dimension? Will the feedback be helpful or hurtful? Will your spaces lead, mislead, or just simply be?
Of course, these are not the only concepts involved in shaping an interreality. There are many many other themes and patterns and behaviors and potential applications in this new liminal reality that are yet to be explored; for the more or less unknown future of VR, interreality is one starting point for us to do so.
What’s up next
All the stuff I couldn’t wrap my head around this time. Specifically, here’s a list of the next five areas I hope to explore:
- Audio: It’s an area I purposefully did not investigate, despite its being a ubiquitous player in interreality. Because it is such a crucial player in designing interrealites, I think it warrants its own line of investigation.
- Augmented reality (AR): I am excited to try to apply these principles to AR experiences. I’m not sure how I will do this quite yet. To me, interreality seems more immediately applicable to the design of augmented virtuality, but that only strengthens my feeling that it can augment conventional reality too.
- Communication: After gaining a better understanding of the effects of audio on interreality, I would like to dive deeper into the field of cross-reality communication. What new modes will emerge that bridge the realities? What does an interreality secret handshake look like? Perhaps this would make a good direction for the next workshop.
- Asynchrony: Can an interactive message be left in interreality for someone to find later? Or does it require the live presence of interreality actors?
- Object permanence: This direction is inspired by both by asynchronous communication the persistence of digital objects. Traditional social media leaves a permanent record of our behavior; can interreallity, (as a digitally mediated social practice itself), afford the same permanence?
For a reiteration of this article and more gifs and a bunch of observations and more experiments forthcoming, you can visit this site.