Design for Good or Evil
Virtual Reality: The Good, Bad, and Ugly
The User Experience of Virtual Reality (Part 2)
Introduction
“Cyberspace consists of transactions, relationships, and thought itself, arrayed like a standing wave in the web of our communications. Ours is a world that is both everywhere and nowhere, but it is not where bodies live.”
— John Perry Barlow, Declaration of the Independence of Cyberspace
Cyberspace is very much a new wild west, at least that’s how John Perry Barlow and Barack Obama describe it, and they are not the first to do so. Cyberspace was visualized in it’s earliest days by William Gibson, author of Neuromancer, Count Zero, and Mona Lisa Overdrive all three of which helped manifest the rules of cyberspace. These books are set in a future where the abundance of technology outweighs progress; a world where citizen surveillance is the norm, and so high-tech that you can experience just about anything from the comfort of your home by using the power of virtual reality (VR) to enter cyberspace. In such a society, it is natural that virtual reality becomes the greatest tool for escape and self-expression.
Virtual reality is a tool that can, in many respects, provide a window for visualizing cyberspace in a way that people can understand- typically known as a Metaverse. It can lead to limitless experiences, but is prone to becoming a glitchy, even dangerous, mess. Thus, it requires some form of control in order to protect those who wish to use it productively from those who do malevolently. And there lies a tension: the desires for virtual environments (VE) floats along a spectrum between a chaotic, decentralized wild west metaverse, on one hand, and a metaverse controlled by single or multiple large entities, on the other. Both come with their own baskets of issues.
“Therefore, the metaverse is wide open and undefended, like airports in the days before bombs and metal detectors, like elementary schools in the days before maniacs with assault rifles. Anyone can do anything that they want to. There are no cops. You cant defend yourself, you can’t chase the bad people. It’s going to take a lot of work to change that- a fundamental rebuilding of the whole metaverse, carried out on a planetwide, corporate level.”
— Hiro Protagonist, from Snow Crash by Neal Stephenson
VR is a technology that requires the attention of all of our senses to be the most effective. At a physiological level, our brain and sensory organs dance in a sensorimotor loop in order to make sense of simulated environments. Put simply, the interactions and visual cues in virtual environments need to be designed to be understood in a similar way that people interact with the real world. People need to trust it on a visceral level. All together, it is a technology that begs us to be human, to feel one with it, and perhaps more importantly, to experience it with others.
Personally, I think that the most under appreciated differentiator that VR has over any other medium is how it can separate us from the environment we are in- whether it’s an isolated spaceship, a hospital bed, solitary confinement, the dentists chair, or, as my friend Marty once put it- our crappy living room. I will get into this in more detail in the fourth section of this paper.
What is This Paper?
“Instead of treating VR and related technologies as a replacement for in-the-flesh interaction, we should think of them as providing opportunities for new and perhaps enhanced modes of human interaction.”
— Andy Clark, Natural-Born Cyborgs
This paper is a follow-up to one of my previous articles- “The User Experience of Virtual Reality” which covered the history of VR, Interaction Design, Visual Design and User Research principles for VR at a high level. It is a companion piece to my World Usability Day presentation of the same title and follows this year’s theme of “Design for Good and Evil.”
My goal with “Virtual Reality: The Good, the Bad, and the Ugly” is to take things a step further by introducing good design practices, bad design practices, and design practices that could make VR the most self-enclosing invention ever created. Finally, the paper will cover some of the functions that VR can have for improving humanity- and it’s potential to become the most expansive invention ever created. This article is structured as follows:
- The Good: Good design practices for virtual reality.
- The Bad: Bad design practices for virtual reality.
- The Ugly: An academic review of the dangers of virtual reality.
- The Beautiful: An academic review of the benefits of virtual reality.
There are a few key concepts to cover before getting started: Presence, Immersion and Embodiment. It is also worth mentioning that I will often be using the word “immersant” in this paper. By immersant I mean “a person who is using VR.” It is a substitute for the generic term “user” and was introduced to me by Brenda Laurel in her book “Computers as Theatre”.
Presence and immersion are terms that are very inter-related. One clear distinction between presence and immersion is provided by Slater and Wilbur (1997). They suggest that presence in a VE is inherently a function of the immersants psychology, representing the extent to which an individual experiences the virtual setting as the one in which they are consciously present. Or, as Matthew Lombard defines it- “the illusion of non-mediation”. Immersion on the other hand can be regarded as a quality of the system’s technology, an objective measure of the extent to which the system presents a vivid virtual environment while shutting out physical reality. A working hypothesis in academia is that that higher immersion is correlated with higher psychological presence. (Cummings and Bailenson, 2016)
Embodiment (or sense of embodiment, “SOE”) is a related phenomenon to immersion and presence, where the immersant believes that they are in the body of the avatar. Specifically, a body that feels ‘‘theirs,’’ which moves according to their intentions, obeying their will. Embodiment is also sometimes referred to as the Body Transfer, or the Proteus Effect- which is named after the Greek god Proteus who was able to take many forms. The subjective capability for immersant SOE is known as homoncular flexibility.
“The combination of VR and interoception (awareness of bodily sensations) leads to [what she describes as] “embodied presence”: not only do you feel like you’re in a VR environment, but because you’ve consciously worked to integrate your bodily sensations into VR, it’s a fuller, more vivid version of presence.”
-Peter Rubin, Future Presence
My hope is that this paper can be used by developers, fellow designers, business owners, or those just interested in the medium, to create awesome VR experiences- or just learn a little more about it. So, let’s get started!
The Good
How to Design User-Friendly Virtual Reality
“Second VR Definition: A simulated new frontier that can evoke a grandiosity recalling the Age of Exploration or the Wild West” — Jaron Lanier
Major Takeaways
- Diegetic UI decisions can improve the realism of a simulation.
- Visual, audio, and haptic cues must be used to guide the immersant- drawing focus to what is 90 degrees in front of them.
- Skeumorphic design in VE’s help immersant understand how to interact with objects in a similar way as they would in reality.
- Storytelling, Improv, and Imagineering are three powerful methods used to design virtual reality.
- Use the diegetic, narrative, and agency needs of your personas as well as their emotional needs as guides for designing your experience.
- Emotion is correlated with presence- but designers should not rely on fear to keep immersants engaged.
- Build tension within narratives to keep immersants emotionally engaged.
In order to understand why and when VR should be used, it’s important to understand how to design for VR. Ultimately, the goal of designing for VR, like any system, product or service depends upon the context. For example, designing industrial applications generally doesn’t require much focus to be put on narrative, where consumer or entertainment applications beg for the designer to pull the immersant into the virtual world as much as possible and to provide a higher degree of presence. This section will focus on the latter, and a lot of the data in this section comes from the Interaction Design Foundation’s course on “Interaction Design for augmented, virtual and mixed reality”. Click the link below to check the course out, it’s really interesting!
Diegetics and the Spatial UI
Presence can be improved through the use of diegetics. Diegetics can be thought of as a lever that the designer can use to control the amount of immersion that a simulation or scene is capable of. An example of this is diegetic user interfaces. These are interfaces that are placed in the world in a way that appear natural and pleasing. They blend, and support the immersive nature of the virtual world. A non-diegetic user interface which is popular in things like real-time strategy games, are not in the immersant’s world but overlaid on the immersant’s screen.
A non-diegetic UI is more similar to the interaction that takes place between a gamer and a computer screen, whereas a diegetic UI constrains more spatially interactive elements that bring participation of the immersant into a virtual world. Below is an example of a diegetic UI as it appears in the world of Star Trek Bridge Crew.
“VR software is a product both of the ancient art of story telling and the most advanced applications of digital technology”… “VR is the first significant step on a long road that culminates when technology and people are no longer separated by screens, when reality and computer-generated images become so tightly integrated that you just can’t tell one from another.”
-Robert Scoble & Shel Israel, The Fourth Transformation
The reason I address diegetics is to make the point that the environment or spatial UI can and will have an impact on the immersant’s sense of presence, and overall enjoyment of the simulation. It is worth noting that the process of designing for spatial UI is prevalent in virtual, augmented, and mixed reality.
The spatial UI, which is essentially the way a UI can be distributed throughout an environment, does not necessarily need to take the form of a graphical user interface like one we would find on a mobile phone. An example might be a character in the environment that helps guide the immersant or assists them in completing a goal. If a non-diegetic user interface is needed, try using a radial UI (see the image above). For social VR experiences these can be very useful as a toolset to interact with the environment. Radial UI also make it easier for the immersant to navigate through menus when pivoting a controller. Radial UI simply make for a better feel.
Designing the 360 Canvas
Unlike traditional screens, VR design involves no canvas, since the immersant can potentially interact with 360 degrees worth of content. Most other mediums such as a computer screen or television screen allow directors to focus the viewer’s attention. This is not the case in VR, and designers must instead rely heavily on cues, whether it’s sound or visual cues to keep immersants moving through the narrative. Whereas cues now can seem obvious (such as an arrow pointing in a direction the immersants should turn) the subtlety of cueing will be a huge focal point for VR creators going forward.
Cues, or spatial UI elements, are typically placed within the 90 degrees of the immersants’ vision upon entering the scene, unless of course you want them to search for something. For example, in Batman Arkham VR, the camera is placed in a scene so that the immersants can easily find and interact with the batsuit, which is necessary for them to progress through the narrative.
Overall, the purpose is to help guide immersants, to teach them, and to help them understand how the world works. A lot of the time this is done by relying on what the immersants already know from the real world- for example, how to water a plant. This is why skeumorphic design is making a comeback with VR. People naturally understand how to use things that they have used or interacted with in the real world. In this same regard, there is also a lack of a mouse and keyboard in most VR applications that require the immersants to stand or move throughout their physical space, which means that all navigation and exploration is done spatially.
Agency
Another powerful tool that designers need to consider is agency which is defined by Janet Murray as “the satisfying power to take meaningful action and see the results of our decisions and choices.” The amount of agency that an immersant can have typically falls into three categories:
- No Agency: Popular in 360 film, the immersant is merely a passive participant.
- Local Agency: Popular in narrative-driven simulation where the immersant is an active participant, navigating through different scenes by making decisions. For example, in the games Portal and Half Life, the immersant must interact with the environment to proceed to the next controlled space full of objectives. Local agency can be both high and low depending on the circumstances of a decision.
- Global Agency: The immersant is theoretically “god” and can impact the simulation however they wish.
The designer should decide which level of agency they would like their immersants to have before designing an experience. The level of agency can help to determine constraints for what the immersant can and cannot do in the VE. In many cases, higher agency is related to higher presence. Awareness of agency limitations can remind the immersant that they are in a simulation.
Building the Narrative
Thinking like your immersant, and asking questions such as “What will I get from this experience?”, “Why will I start this voyage?”, “How will I accomplish my goals?”, “Where can this technology take me” and “What can I do there?” can allow us as designers to start to imagine some pretty powerful narratives and desires for levels of agency and diegetics. To do this, it is essential that the designer become great at storytelling, improvisation and a skill known as imagineering.
Storytelling is something everyone has done at least once in their lives. We tell stories to pull people into a journey from beginning to middle to end, introducing sub-plots or drama to keep the stories engaging. Some are better at telling stories than others, and some get paid a lot of money to write stories that can be engaging for hours on end. Stories are entertaining. Ultimately a great story can help to deepen presence. Dramatic form and storytelling can be understood better by reading Brenda Laurel’s work “Computers as Theatre” in which she dissects the dramatic form of human experience as it relates to interaction with technology.
Improvisation is one of the best analog methods of designing for virtual reality because it allows designers to act out and expand upon ideas in space. The reason this is especially useful when designing for VR is because improvising in reality is a low-cost way to act or test out experiences in virtual reality. Participants in improvisation exercises can act our VR scenarios without ever having to interact with a computer. Abstraction, imagination and a little creativity can help designers visualize three dimensional environments with little to no technical requirements.
The third skill is Imagineering- an intentional approach to creating engagement and delight. It’s a skill that allows designers to invision things or interactions that are beyond capabilities in the real world- opening up new pathways to possibility, similar to speculative design. Imagineering is the magic that turns the ordinary into extraordinary, ultimately providing awe and wonder and making VR worth it. The term was popularized by Disney who created the “Triangle of Imagineering”.
The triangle involves four entities including:
- The Dreamer (Visualize any scenario)
- The Realist (Map practical details e.g. path pros/cons)
- The Critic (investigate risks, why it will/will not work)
- The Observer (storyteller and facilitator).
The Triangle of Imagineering exercise not only pulls together different perspectives from different people, but provides a framework for idea development and innovation in VR- imagining magical new realities. It is a tool that VR designers can use to add some magic to their experiences.
Final Points on Designing Good VR
Like most experiences, “Less is More” applies to VR just as much as anywhere else. It applies to discovery, menu density, movement, visual input, and cognitive load. When designing for VR, it is extremely important not to stress out or over-excite your immersants. Keep your personas in mind, and how they should adapt to the context of the virtual experience as well as the tasks they are provided with. There is nothing wrong with creating an experience that fosters joy and pleasure. Scaring, or thrilling your immersants is simply cheap and easy to do. Consider this quote from psychologists Barbara Fredrickson and Thomas Joiner about positive emotions:
“Positive emotions broaden people’s thought-action repertoires, encouraging them to discover novel likes of thought or action. Joy, for instance, creates the urge to play, interest creates the urge to explore, and so on. Play, for instance, builds physical, socioemotional, and intellectual skills, and fuels brain development. Similarly, exploration increases knowledge and psychological complexity.”
(Emotional Design by Donald Norman)
The goal of the previous section was to introduce some of the considerations designers should have for creating VR experiences. Now that we know the basic concepts needed to design in VR, let’s dive into what happens when we put these concepts into the wrong hands…
The Bad
How NOT to design for Virtual Reality
“We think VR is going great. It’s going in a way that’s consistent with our expectations. We’re also pretty comfortable with the idea that it will turn out to be a complete failure. Simply because, if you aren’t trying to do something that might fail, then you aren’t probably doing anything very interesting at all.”
— Gabe Newell, Valve
Major Takeaways
- Overwhelming the immersant’s senses, or placing too much emphasis on a single sense can not only break presence but cause simulation sickness.
- When phenonema in the virtual environment “lag”, it can cause immersants to experience ocular-vestibular mismatch- resulting in simulation sickness.
- Acceleration (both angular and translational) or turning an immersants camera or viewport in un-natural ways can cause ocular-vestibular mismatch, which will cause simulation sickness.
- Non-immersant initiated vertical or horizontal movements, moving them without their permission or in unexpected ways can be jarring.
- Balance comfort and realism to protect the immersant’s sense of presence.
- Holograms placed too close to the immersant can induce discomfort especially when combined with abrupt movements.
- Designers of immersive experiences can become biased to the effects of their own simulations.
I remember listening to Gabe Newell speak about virtual reality with Valve News Network. One of the most profound things he mentioned was that VR is not going to be a success if people are just taking existing content and putting it into a VR space. Take, for example, any old Nintendo game (e.g. Super Mario 64) into VR without considering how or why the game should be in VR. Newell also mentioned that noone is going to buy a VR system just so they can watch movies. Gabe’s hope is that VR will allow us to find potential in the medium beyond anything that is possible on any other medium.
With VR being in such a young state (commercially), immersants are extremely sensitive to poor design decisions. There is anticipation in trying the hardware for the first time, not to mention the software. Though there is academic research that suggests people are almost usually delighted by trying VR for the first time (Tussyadiah et al. 2007), there is a massive body of research showing how this delight can quickly turn to regret — especially if the immersant loses interest within the first 30 seconds of trying a simulation.
Some of the main reasons for users losing interest in a simulation, as outlined by the Interaction Design Foundation are: Overwhelming or underwhelming experiences, poor hologram placement, imbalance between comfort and realism, and lack of empathy for new immersants.
Overwhelming Experiences
Overwhelming an immersant can have lasting and even traumatic effects. I remember watching a simulation of Neil deGrasse Tyson performing a presentation in VR. I was able to get really close to the hologram of Tyson. In one part of his presentation about the concept of hell, a demonic imp crawled out of the wall right in front of me. I was paralyzed, for a moment, in terror. I felt very present, for sure, but also very uncomfortable and scared. That being said, I was emotionally engaged, and therefore, present. Breaks in presence (caused by poor latency, rendering or tracking for example), or just an emotionally dull experience, can break the narrative experience.
Since VR demands control over the immersants’ senses, it is the designers responsibility to treat the immersant with even more respect than any other medium. I think that this video, Hyper Reality by Keiichi Matsuda, paints a pretty good portrait of an overwhelming experience using mixed reality.
Not only is it visually overwhelming, but puts the immersant in danger as well as their identity is stolen which opens up another bag of ethical issues.
Mind the Senses
There is a lot of research from Oculus that shows overloading one sensory system — for example, using only speech, touch, or vision to help an immersant proceed throughout an experience — is a poor design practice. People perform better in VR with multi-sensory UI’s — that is, an experience that takes advantage of all of the their senses rather than dumping all interaction into one. Otherwise, the simulation will feel artificial.
The same can be said for hologram placement. According to Microsoft Hololens guidelines, Holograms in VR should be placed in a 1.25-2.25 meter focal plane. If a hologram is too far away it feels too unrealistic or uncomfortable to the immersant. Too close, and it can startle or make the immersant feel claustrophobic. In this regard, the laws of proxemics and ergonomics apply equally in VR as they do in the real world.
Comfort vs. Realism
A big argument I mentioned in my previous article “The User Experience of Virtual Reality” was the tension between providing a comfortable versus a realistic experience in VR. It’s no surprise that many early VR designers worked hard to create the most realistic experiences as possible, considering the link between immersion and presence. However, this can come at a cost especially without the three major ingredients required for a quality VR experience — optimal rendering, tracking and latency. The costs can be sickness and eyestrain, which can lead to very poor experiences in VR.
Take, as an example, locomotion, meaning the way people move around a simulation. Locomotion on its own is the cause of much frustration in the VR community- mostly because hardware constraints make it impossible for people to move naturally (there’s a lot of work being done on this issue). Unrealistic locomotion, and people don’t believe it — realistic locomotion, and people get sick. This is typically due to ocular-vestibular mismatch where our brain cannot make sense of it’s position in the environment in a way that makes sense- a sense that is referred to as proprioception.
— Yasser Malaika, VR Interaction Designer, Valve Software (Photo sourced from RoadToVR)
Some Final Points on Bad Design Practices
“In five years from now, if the people in this room are putting on helmets to read their email, then I have failed as an evangelist.” — Jeremy Bailenson
According to the Interaction Design Foundation, there are a few other major things to avoid when designing for VR. A lot of this has to do with how the immersant is oriented in the simulation. For example, if they are using 6 degrees of freedom (X, Y, Z, roll, pitch, and yaw), then don’t shake the camera or purposely lock it. It can make people feel uncomfortable as it’s motion that was not initiated by themselves.
In order to prevent immersants from becoming sick, designers need to test their simulations on people other than themselves. It’s easy to become tolerant to experiences that we create, even if they make us sick at first. Ocular-vestibular mismatch and misaligned proprioception might seem like non-issues to someone who has tested a simulation numerous times. VR is completely different than any other medium, and most people haven’t experienced it yet. Ease them into it, don’t overwhelm them.
So that covers good and bad design decisions for virtual reality. The next two sections are going to cover academic research surrounding the darker, evil side of VR, as well as the beautiful, profound side that can make a real difference in this world.
The Ugly
The Dangers of Virtual Reality
“Virtual reality, in order to work well, would have to include the best possible sensing of human activity ever. And it could create practically any experience as a form of feedback. It could turn out to be the evilest invention of all time.
-Jaron Lanier
Major Takeaways
- Long-term immersion in VR can lead to addiction, dissociation and fatigue which can lead to the neglect of ones own body, their interpersonal relationships, and their personal space which can (and has) lead to fatalities or chronic illness.
- Poorly designed or meaningless narratives or social interactions can have a long-term effect on an immersant’s mental health and agency.
- Risky content such as that which is violent, extreme, or overwhelming, for example, training someone how to program a nuclear missile, is extremely unethical due to the plasticity of the human mind inside of VR.
- VR affords the opportunity to measure the entirety of an immersants bio-metric feedback which can cause a serious invasion of their privacy.
- Long-term isolation has been associated with lower mental health, which will most likely lead to psychological diagnoses related to overuse of VR.
- Experiencing intense emotions inside of VR may have lasting, or even traumatic, effects outside of VR. Because of how immersive VR is, effects can last outside of wearing the headset and into reality.
In the last several years, a number of studies have found a psychological influence on subjects while immersed in a virtual environment. These studies suggest that VR poses risks that are novel, that go beyond the risks of traditional psychological experiments in isolated environments, and that go beyond the risks of existing media technology for the general public.
In 2016, Michael Madary and Thomas Metzinger published in article in the journal Frontiers in Robotics and AI titled “Recommendations for Good Scientific Practice and the Consumers of VR-Technology”. A lot of this next section will explore some of their findings and recommendations.
“The comprehensive character of VR plus the potential for the global control of experiential content introduces opportunities for new and especially powerful forms of both mental and behavioral manipulation, especially when commercial, political, religious, or governmental interests are behind the creation and maintenance of the virtual worlds.” — Madary and Metzinger, 2016
When I read the quote and look at the image above, I immediately think about the genre of Cyberpunk. Characterized by abundance of technology over progress, and deep seeded in the anxiety of technological dependence and government control, Cyberpunk is a very dystopian high-tech look at a future where technology can be used to control, persuade, or manipulate people. The closest thing we have today in digital design are a series of patterns known as “dark patterns”. Found in many modern applications, dark patterns ignore human-centered philosophy and are rooted in deception and coercion- typically done for things like capturing unnecessary data or pushing people to purchase something or take a political stance.
Jeremy Bailenson mentioned in Experience on Demand five major risks for Virtual Reality including: Distraction, addiction, simulator sickness, medium modeling, and effects of VR on Children. The rest of this paper will merge Bailenson’s risks with Madary and Metzinger’s four major risks associated with virtual reality:
- Long-term immersion- addiction, dissociation, fatigue
- Neglect of embodied interaction and the physical environment — allowing people to neglect themselves, others, or their physical environment
- Risky content — Content that is violent, extreme, or overwhelming, for example, training someone how to program a nuclear missile
- Privacy- the loss of identity or disrespect of privacy, for example, placing someone inside of an experiment without their consent.
Madary and Metzinger’s risks align well with Bailenson’s. The rest of this section will go into more detail about these four risks and some of the research that has been done to support them.
Long-Term Immersion
Jeremy Bailenson mentioned three main issues that arise from the overuse of VR: Physical sickness, eyestrain and reality blurring. We have already touched on the first two, but the third one raises a major ethical dilemma. In his book Experience on Demand, Bailenson recounts a 2014 study by a team of German scientists from the University of Hamburg. One of the scientists spent 24 hours in VR and was then tasked with separating artifacts from the real and virtual world. He told the scientific VR community that “Several times during the experiment, the participant was confused about being in the VE (virtual environment) or the real world.” This finding alone is troubling when considering the implications of VR addiction.
The biggest issue is that we simply do not know the psychological impact of long-term immersion. So far, most scientific research using VR has involved only brief periods of immersion, typically on the order of minutes rather than hours. Once the technology is adopted for personal use, there will be no limits on the time immersants choose to spend. This is especially troubling when considering recent research by Ketaki Shriram which suggest that people inside of VR are less likely to recognize stimuli outside of VR, for example, something touching their hands. Translate this to neglect and VR could stop people from eating or sleeping.
A major problem discussed by Madary and Metzinger is that VR technology could be used to manipulate an immersant’s sense of agency. That is, their perception of how much control they have over a situation. Within the general context of mental health, long-term immersion could cause low-level, initially unnoticeable psychological disturbances involving a loss of the sense of agency for one’s physical body- or to be more specific, VR dependency.
“With little exposure to “higher” culture, to great works of art and literature; and without the skills (and maybe the attention spans) to enjoy them; people would be less able to engage with the world at a deep level. People without exposure to great works and ideas might find that [their] inner lives are shaped to a large degree by market-led cultural products rather than works of depth and profundity. — Fiachra O’Brolcha (2015)
I think that two of the biggest anxieties surrounding VR are addiction and dependence. The medium is an experience generator with limitless capabilities. It can be used to do the impossible, but can also be used to addict for commercial gain. Think if the way freemium applications work today, where users can engage with them as much as they would like at the price of their time- subjecting them to a barrage of advertisements- essentially brainwashing them.
The book Brave New World by Aldous Huxley painted a picture of immersive media that was completely based on sensory stimulation. Called the “feelies”, the films would provide a sensory stimulus across all of the immersants senses to the point of complete dissociation with reality. Immersant could feel the sensation of a kiss, touch, or just about anything else. Thus, media producers focused on pushing out the most addicting content that would keep people coming back for more. Another novel, Infinite Jest by David Foster Wallace dives into this type of reality even further by suggesting a media so addicting that it is not only banned, but completely lethal due to it’s addictive content.
If VR is built to be addicting, and to demand the attention of it’s immersant (such as how mobile apps are developed now), chronic dependency and addiction could very well become a reality. Long-term immersion in VR might also be unedifying, making people more shallow as they retreat from society in favor of an artificial social world in which their decisions are made for them. This was a major theme in Janet H. Murray’s book “Hamlet on the Holodeck” which challenges the nature of virtual narratives and how people engage with virtual experiences.
Neglect of Embodied Interaction and the Physical Environment
“Case fell into the prison of his own flesh.”
― William Gibson, Neuromancer
It doesn’t take much looking to find the consequences of addiction and neglect especially when it comes to one’s physical bodies, interpersonal relationships, or physical environment. In Neuromancer, the protagonist Case’s addiction to speed can only be paralleled with his addiction to cyberspace. Throughout the book, he even refers to his body as simply “meat” and all of the human limitations that come with it.
Within the past year, we have already seen the first documented VR-related death where a 44-year-old Moscow resident died after falling through a glass table while inside of VR. It goes without saying that as immersant’s spend increasing time in virtual environments, they are at risk of their neglecting their own bodies and physical environments — just as for many people today posing and engaging in disembodied social interactions via their Facebook account has become more important than what was called “real life” in the past. In extreme cases, individuals refuse to leave their homes for extended periods of time, behavior categorized as “Hikikomori” by the Japanese Ministry of Health.
When entertainment numbs people from taking care of themselves and their environment and is used as an escape or a distraction from real life, it can cause humans to not only be lonely, but numb to human interaction. A recent finding from Teo et al. (2015) suggests that depression is more likely in older adults who have less social contact in person regardless of their amount of telephone, written and email contact. Perhaps more importantly, there is a concern that mediating technologies will not allow us to pick up on all of the subtle bodily cues that appear to play a major role in social communication through unconscious entrainment (Frith and Frith, 2007), cues that involve ongoing embodied interaction (Gallagher, 2008; de Jaegher et al., 2010). To put it simply, people could forget body cues that exist in the real world after spending too much time socializing in VR- impacting ones ability to engage in real-life, human-to-human, communication.
Disclaimer: This is not to suggest that social interaction in VR is bad. Social interaction and friendships in videogames, for example, have been shown to reduce social anxiety and produce strong relationships akin to real life friendships. It’s just that balance between interaction in virtual and real environments is necessary. In Peter Rubin’s book “Future Presence” he recalls the story of a group of friends in the game Rec Room, two of which got married in real life.
Madary and Metzinger suggest designers consider how well social interactions VE’s will translate into real world behavior. Shallow or even meaningless interactions (think of today’s Facebook-“friendships” and “likes”) are experienced as substantial by users. The issue is that a shallow form of social interaction could become culturally assimilated and thereby “normalized” in society (Metzinger and Hildt, 2011, p. 247) which could lead to happiness inside the virtual world, depression in the real world, and most of all- addiction and dependence on virtual environments.
Risky Content
“Will the stories brought to us by the new representational technologies ‘mean anything’ in the same way that Shakespeare’s plays mean something, or will they be ‘told by an idiot’?” — Aldous Huxley (sourced from Hamlet on the Holodeck)
There’s a pattern that has been emerging from the dawn of broadcasting media to modern technology. That is, the plasticity of the human mind. To be more specific, I am referring to the ability for media to alter our behavior or thoughts. As Jeremy Bailenson and his colleagues like to put it “media experiences are like a diet: you are what you eat”. This phenomenon is referred to as media modelling and it has to do with the way media experiences impact human behavior in the real world. This is especially troubling when considering the brains of children which are in such rapid development.
So what happens when people are learning how to put together a bomb in VR, hurting people, or damaging property inside of VR? During his trial, the mass murderer Anders Behring Breivik described hours of practice using a holographic sight in the popular first-person shooter Call of Duty: Modern Warfare as “training” for his horrific rampage on the island of Utoya on July 22nd, 2011. (sourced from “Experience on Demand”, 2018). With VR being generally more capable as an educational tool than control conditions (to be discussed later), what are the implications for risky content?
There’s a big rule in the virtual reality community that VR is for things that you couldn’t do in the real world, not what you shouldn’t do. The problem however is that people are naturally curious and want to use technology as a means to experiment with doing the impossible or crossing the lines without consequence. In many respects, that’s one of the capabilities of VR that makes it awesome. A second possible problem is that this rule of thumb would make VR even more subjectively real. That is, limiting opportunities could constrain the amount of magic and potential that VR can provide.
I think it goes without saying that some types of content (which are already banned in video games) should not be allowed in virtual environments. Some obvious candidates that Madary and Metzinger suggest are sex (virtual pedophilia, virtual rape) and violence. I would push it further to say that human torture, animal abuse, and self-harm should be included in this as well. However there is also a sect of content known as the “dark triad” (Paulus and Williams, 2002) that should be considered. The dark triad refers to narcissism, machiavellianism, and psychopathy. Individuals may find it appealing to spend time in virtual worlds designed to reward characters that exhibit traits associated with the dark triad. For example, the MMORPG EVE Online is known for fostering a style of play that involves manipulating and deceiving other players. The issue being that engaging in these behaviors could condition someone to become more comfortable with doing them in real life- a hypothesis supported by findings in behavior modification through VR (to be discussed).
“Today, one of the biggest problems for virtual reality is that the immediately obvious customer base willing to spend money is gamers, and gaming culture has been going through misogynist convulsions.”
- Jaron Lanier
More recently, an experiment reproducing the famous Milgram obedience experiments in VR found that subjects reacted as if the shocks they administered were real, despite believing that they were merely virtual (Slater et al., 2006). These findings suggest that even in virtual environments, people will obey an authority figure even if it means committing an atrocity.
Privacy (Skinner Boxing)
Thirteenth VR DefinitionL The perfect tool for the perfect, perfectly evil Skinner box. -Jaron Lanier
A Skinner box is a controlled environment where scientists can take accurate measurements from a subject as they participate in an experiment. In VR, this can very much be done without the participant even knowing, if you think about how much data is being tracked about a person at a time.
Perhaps the most ugly side of virtual reality is associated with control. One movie I like to think of that captures this anxiety is the Matrix trilogy. Based around Simulation Theory, the film follows protagonist Neo as he uncovers the truth behind humanity’s existence. The truth is that all humans are stuck in an AI-controlled simulation designed to keep them numb, distracted, and satisfied to the point that they are completely unaware that what they are in is a virtual construct.
Don’t let a technology manipulate people on the behalf a third party- that’s the bottom line from one of Jaron Lanier’s recent talks in the metaverse Sinespace. He goes on to say:
“if you think about what VR is, it’s measuring a lot about a person. You have to have accurate motion tracking, facial expressions, and all kinds of things. Then you have the feedback that the machine captures. So you can theoretically build a Skinner Box. VR is good for anything, not just connecting people, but could also be used for behavior modification. To what degree is it about making a connection, and another about manipulation and trickery? The technology is good at both. The components of a Skinner box and those of a cybernetic computer are essentially the same.” — Jaron Lanier
One of the problems and issues that Madary and Metzinger address is- what happens when commercial advertising is prioritized over human privacy? Jaron mentioned in his previously mentioned talk how disturbing it was for him that Facebook purchased Oculus. The reason being that Facebook has already allowed corporations to push their ads to the masses in a really irresponsible and uncontrollable way, which ultimately lead to coercion and deception which are the hallmarks of unethical persuasive technology.
As we’ve spoken about in this article already, VR allows people to experience and do things like never before which can lead to behavior modification. This is because our brain treats VR in a similar way that it does reality. Commercial applications of virtual environments introduce new possibilities for targeted advertising or “neuromarketing,” thus attacking the individual’s mental autonomy. By tracking the details of one’s movements in VR, including eye movements, involuntary facial gestures, and other indicators of what researchers call low-level intentions or “motor intentions” (Riva et al., 2011), private agencies will be able to acquire details about one’s interests and preferences in completely new ways (Coyle and Thorson, 2001).
“Thirty-eighth VR Definition: The ultimate way to capture someone inside of an advertisement. Let’s hope it’s done as little as possible.” -Jaron Lanier
Disclaimer: This is not to suggest that this is what Virtual Reality WILL become. It is just saying that it is what it COULD become if the end user is not prioritized over commercial giants.
Commercials in VR could even feature images of the target audience himself or herself using the product. The use of big data to “nudge” users (“Big Nudging”) combined with VR could have long-lasting effects, perhaps producing changes in users’ mental mechanisms themselves. Immersant’s ought to be made aware that there is evidence that advertising tactics using embodiment technology such as VR can have a powerful unconscious influence on behavior. For example, VR has had influence on behavior in controlled settings, making subjects willing to save more for their retirement (Hershfield et al., 2011), perform better on tests for implicit racial bias (Peck et al., 2013), and behave in a more environmentally conscious manner (Ahn et al., 2014). If individuals do not seek to alter their psychological profile in the ways intended by the beneficent VR interventions, then such interventions may be considered a violation of their autonomy.
Changes in behavior while in the virtual environment are of ethical concern, since such behavior can have serious implications for our non-virtual physical lives — for example, as financial transactions take place in a non-physical environment (Madary, 2014).
Academic Case Study — Emotions inside of Virtual Reality
What is historically new, and what creates not only novel psychological risks but also entirely new ethical and legal dimensions, is that once VR gets ever more deeply embedded into another VR: the conscious mind of human beings, which has evolved under very specific conditions and over millions of years, now gets causally coupled and informationally woven into technical systems for representing possible realities. — Madary and Metzinger
This short section is going to weigh a little heavy on the academic side.
If one theme has resonated through this paper I hope it’s that VR is for things that we couldn’t do in reality, not what we shouldn’t do. VR has the ability to make us feel, it’s this emotion that it provides that provides the illusion of presence. It can be something that makes us feel fear, anger, sadness, joy, disgust, surprise, trust, anticipation among any other emotion the medium allows us to provoke.
Virtual Pit studies show that people are prone to real and perceived physical and social dangers inside of Virtual Reality including strong feelings of stress and fear. In addition to a strong emotional response from immersion, there is evidence that experiences in VR can also influence behavioral responses. One example of a behavioral influence from VR has been named the Proteus Effect by Nick Yee and Jeremy Bailenson. This effect occurs when subjects “conform to the behavior that they believe others would expect them to have” based on the appearance of their avatar (Yee and Bailenson, 2007, p. 274; Kilteni et al., 2013). They found, for example, that subjects embodied in a taller avatar negotiated more aggressively than subjects in a shorter avatar.
When experiencing anxiety, people report heightened physiological arousal (e.g., heart rate, blood pressure). Research from Levin et al. (1993) showed that social anxiety reveals itself in ineffective self-presentation behaviors. These include behaviors such as trembling, fidgeting, stammering, or stumbling over words. Leary et al. (1995) found these behaviors create negative impressions on others. Research from Gregg et al. (2007) and Opris et al. (2012) showed the effectiveness of VR as a treatment for anxiety disorders.
Research from Matilda Annerstedt (2013) suggested that social anxiety stimulated inside of VR via Trier Social Stress Test (TSST) can be attenuated with visual imagery and sound (specifically, a virtual forest). Annerstedt also noted heightened indicators of anxiety even up to 40 minutes after anxiety onset in a condition with a virtual forest without sound. It was suggested that this was due to anticipatory anxiety inside of VR. The viability of anxiety treatment in VR is based on the observation that virtual environments can elicit similar subjective and physiological reactions as real situations (Cornwell, Johnson, Berardi, & Grillon, 2006; Diemer, Mühlberger, Pauli, & Zwanzger, 2014; Mühlberger, Bülthoff, Wiedemann, & Pauli, 2007; Mühlberger, Petrusek, Herrmann, & Pauli, 2005; Villani, Repetto, Cipresso, & Riva, 2012).
Allcoat, Greville, Newton and Dymond (2015) ran a classical conditioning study examining the physiological effects of an anxiety-provoking stimulus (a 90db female shriek) in virtual reality. They found reliable suppression of operant behavior by a Pavlovian threat cue embedded within a VR task which mimicked the freezing-like responses that often occur in the presence of aversively learned cues in individuals with anxiety. Thus, it can be deduced that VR can trigger real stress and physical reactions in immersants.
Due to the isolating nature of virtual reality, social interactions in virtual settings, and the best way to moderate them, is a well-known issue in the VR community. Kipling Williams (2000) famously created the game Cyberball to demonstrate ostracization, prejudice and discrimination with statistical significance.
Williams and Jonathan Gerber (2005) have shown in an fMRI that ostracism in VR causes areas associated with pain and sadness to light up. Beatrice Hasler, Benhard Spangland and Mel Slater (2017) extended these experiments into virtual reality and found data that suggest people may behave negatively toward an outgroup when an intergroup threat is indirect. Research from Anderson-Hanley, Snyder, Nimon and Arciero (2011), Nunes, Nedel and Rosler (2014) and Snyder, Anderson-Hanley, and Arciero (2012) suggests that the effect of a competitor in VR has been examined to produce performance enhancement, particularly for those who report being more competitive. This body of research on social VR suggests that the phenomena of competitive drive, discrimination, prejudice and ostracization may take place in VR and thus, induce excitement, stress, or even depression on it’s immersants.
To put it simply- experiencing intense emotions inside of VR may have lasting, or even traumatic, effects outside of VR.
The Beautiful
The Benefits of Virtual Reality
“VR is an amazing technology, the brain treats it in a similar way to being real, it’s something that can take us to different places, it’s powerful. We should save it. Save it for things in the real world that meet one of four standards: Impossible, Counterproductive, Rare/Expensive, or Dangerous”
— Jeremy Bailenson, Talks at Google
Major Takeaways
- Virtual reality provides a vessel for human-to-human connection and interaction beyond any technology we have seen before, allowing us to battle the human-centered issues surrounding isolation.
- Virtual reality can be harnessed for behavior modification and rehabilitation for both good and evil. It affords the opportunity to provide cognitive-behavioral therapy for any addiction, trauma, or phobia.
- Virtual reality can be harnessed for distraction from the outside world, especially for distraction from chronic pain, or unpleasant stimuli (e.g. ER)
- Virtual reality can be harnessed as a tool for empathy building, allowing immersants to embody something other than themselves and experience the world from a different perspective.
- Virtual reality can create any experience, allowing us to experience things that are impossible, counterproductive, rare, expensive, or dangerous in the real world.
- Virtual reality can be harnessed for education and training both in schools and industrial sessions. It allows companies to save on expensive resources or scenarios that are dangerous for their employees.
- Virtual reality affords the opportunity to make the boring more exciting by allowing immersants to experience things (such as exercise) in new or exciting ways.
VR has changed the way I look at people and the world. I truly believe that it’s something everyone on this planet should try at least once. It’s predicted that mass adoption will take place once the hardware and software reach a certain level. Companies like Valve, Sony, Facebook, Microsoft, Magic Leap, Snap, Google, Pimax, (and potentially Apple and Nintendo based on recent rumors) are investing billions of dollars into the future of virtual, augmented and mixed reality… for better or worse.
The final piece of the paper will focus on the beautiful sides of VR, and will address just a few of many major findings that suggest VR as a transformative tool. Things that could provide good for humanity. I will focus on three subjects at a high level: Human Connection, Rehabilitation, and Empathy Building.
Human Connection
While the technology can seem cool or surprising in lots of different applications, interpersonal VR experiences often feel deeper, truer, than conventional social media platforms or multiplayer games. — Peter Rubin, Wired
There’s a popular belief related to VR that has resonated throughout the past decade or so: Instead of treating VR and related technologies as a replacement for in-the-flesh interaction, we should think of them as providing opportunities for new and perhaps enhanced modes of human interaction. A place where two people can interact with each other, see anything, and do anything they can imagine together. VR gives us the ability to take normal interactions in the world and provide superpowers for them, making something as simple as a high-five feel like a powerful splash of human connection. This social phenomenon is something that was recently referred to by Mark Zuckerberg as co-presence, the overwhelming sense in VR not just of being somewhere real, but being there with another person.
If you’re interested in learning more about co-presence in VR Chat, check out my article above.
Wired editor Peter Rubin goes into great detail about social VR in his book Future Presence and how VR is able to provide social connection in a way unlike we’ve ever experienced. It allows us to socialize under the veil of anonymity, to create social groups, or to build lasting friendships in the virtual world. He goes to say:
“Many argue that mobile smartphones and social media have made us less connected to our fellow human beings. VR has the potential to course-correct the isolating nature of much of today’s technology and the opportunity to make us more connected and even more human.” — Peter Rubin
In a recent interview with the University of Pennsylvania, Rubin answered a question related to the concept of intimacy in VR.
Knowledge@Wharton: “Intimacy is part of your book title. Can you talk about that?”
Rubin: “A chief ingredient of the intimacy [fostered] in virtual reality begins with the fact that you can have eye contact. As eye tracking gets into headsets in the next year or so, we will have the ability to have our gaze mirrored in virtual reality, everything from blinks to winks to where we are actually looking. We can already make eye contact with people, but when you bring in even more naturalistic cues, that creates the ability to turn what happens in VR into something that’s much more like real life.
It’s not like looking at a photo. It’s not like anything we’ve ever experienced before. If you have a real-life memory of something you’ve done in VR in a fantastic environment — whether it’s something as pedestrian as being under a starry sky, or you’re on the surface of Mars, or you’re floating over Central Park — your memory is still spending quality time with this other person or these other people in this magical place.”
Rehabilitation
“Thirty-sixth VR Definition: A way to try out proposed changes to the real world before you commit.” -Jaron Lanier
My sister is studying to be an Occupational Therapist. That is, someone who works to help people learn how to use their motor skills so that they can participate in society. She and I were talking about how they use VR to reinforce voluntary repetition, which is a key ingredient for motor recovery based on principles of neuroplasticity.
Earlier this year I wrote a proposal and literature review for a Master’s thesis surrounding the topic of social anxiety and virtual reality. Some of it is featured in the above section “Academic Case Study — Emotions inside of VR” . My hypothesis was that the physiological response to social anxiety could be attenuated with repeated exposure to social support in VR. I came up with this idea after learning about a lot of research being done to help people who struggle with a number of disorders. Tyler Rose, Chang Nam, and Karen Chen’s 2018 literature review “Immersion of virtual reality for rehabilitation” covers a lot of these disorders. Their findings suggest that VR can be used to help treat PTSD, arachnophobia, strokes (Jack et al., 2001), cerebral palsy (Reid, 2002), severe burns (Haik et al., 2006), Parkinson’s disease (Mirelman et al., 2010), Guillain-Barré syndrome (Albiol-Pérez et al., 2015), and multiple sclerosis (Fulk, 2005), among others. VR also offers the ability to customize treatment needs while delivering increased adjustment of assessment and training procedures (Sveistrup, 2004).
The following section is going to outline some case studies about how the power of VR is being used to assist those who struggle with overcoming isolation, fear, and pain in the real world.
-Isolation-
NASA has funded scientist Peggy Wu of the research company SIFT to study how VR could provide psychological support for astronauts on deep space missions. SIFT’s program — which is called ANSIBLE — contains virtual worlds on which astronauts could visit art galleries, nature preserves, and environments similar to home, or interact with avatars of their friends. The purpose of this is to rehabilitate them to human connection, to prevent loneliness, and to provide contact with their loved ones in a virtual space.
NASA also plans to use VR as a vehicle for training astronauts for space, as well as life on Mars. Replacing the expensive equipment usually required for training with digital holograms that can help trainees move through the motions of maintaining a space shuttle among other various exercises.
I think virtual reality is a communication channel, just like real reality. We have more capabilities to manipulate VR, which to me, means we are unencumbered by physical limitations to learn about and make meaningful connections with other people. — Peggy Wu
This sheds light onto one of Bailenson’s four uses for VR which is to do the impossible. To foster a connection between people that are thousands of miles apart through nothing more than illusion.
-Fear-
In collaboration with others, Barbara Rothbaum (a clinical psychologist from Emory) and Larry Hodges (a computer science expert from George Tech) were the principle investigators in the first published Journal study on using VR exposure therapy for treating a phobia (fear of heights, see http://www.cc.gatech.edu/gvu/virtual/).
Since then, Rothbaum and Hodges have had great success using VR exposure therapy to treating fear of flying, and they have had some encouraging preliminary results treating post-traumatic stress disorder in Vietnam Vets, a disorder notoriously difficult to help (unlike phobias, which are easy to treat quickly and successfully). Hodges and Rothbaum are currently exploring the use of VR for treating fear of public speaking as well. Several of the virtual worlds developed by Hodges and Rothbaum are now commercially available for clinicians interested in using VR exposure therapy with their patients (see http://www.virtuallybetter.com/).
To reach back to Bailenson’s uses for VR, treating phobias with virtual reality helps people do things that are dangerous. For example, standing on a ledge thousands of feet above the ground. To push things further, this is behavior modification for good, as the client is completely aware that they are participating in the simulation with the end goal of overcoming a fear. A good commercial example of this is “Ovation” which allows immersants to practice presenting in front of crowds without the risk of social judgment.
In the immersive virtual world called SpiderWorld, patients can reach out and touch a furry toy spider, adding tactile cues to the virtual image, creating the illusion that they are physically touching the virtual spider. (Hoffman, 2001) Tactile augmentation was shown to double treatment effectness compared to ordinary VR. The topic of tactile and haptic feedback is a bit outside of the scope of this paper, but combining physical elements with virtual elements can increase presence even further.
-Pain-
As previously mentioned in this article, distraction inside of VR can be perceived as a double-edged sword. On one side, distraction can mean forgetting to take care of one’s physical body or environment. On the other side, distraction can mean forgetting or being unaware of extreme pain or terrible surroundings. One thing is for sure, research has shed light on both sides, and VR has the capability to distract people from pain like no technology that has ever been seen before.
To experiment with VR for pain distraction, a number of research groups have experimented with different VE’s that can help distract burn patients from the painful procedure of removing their bandages. At it’s earliest stage this was called Snow World, and the latest iteration is called Mobius Floe.
The designer’s goal for SnowWorld was to distract immersants from pain while keeping their constraints in mind. In many cases, the immersant was completely constrained. Thus, locomotion needed to be automatic or at least simple because they cannot use their hands or feet to maneuver themselves through the VE. In many cases, the patient is also sedated, so tasks need to be simple enough as not to distress the patient, but also complex enough to distract them from the outside world.
Initial trials have been promising. In a small clinical trial featuring 40 people — each receiving 60 VR sessions — all but one reported reduced pain. Overall, patients reported 60 to 75-percent less pain than before their VR sessions. Immediately following a single session, patients reported 30 to 50-percent less pain. For comparison, morphine averages around 30-percent pain reduction. This research has implications for battling the opioid crisis which suggests VR may be able to distract people from pain more than opioids alone.
Empathy Building
“Twelfth VR Definiton: VR is the technology of noticing experience itself.”
-Jaron Lanier
One of the main themes in Jeremy Bailenson’s Experience on Demand was that VR can be used as an empathy-building machine. The basic gist is that through the phenomena of embodiment, immersant’s can take on the persona of whatever avatar they are representing in VR. Or, as Bailenson puts it, “body transfer states that if you move physically and your virtual avatar moves synchronously with you (you see it through a mirror, or in first person), then over time the part of the brain that includes the schema for the self expands to include that virtual representation.” This causes someone to literally feel like they’ve become someone else- a different gender, a different age, or a different skin color, or even — a different species!
The implications for this are that VR can act as a sort of empathy machine. It allows people to feel what it’s like to walk in someone (or something) else’s shoes. To push it further, a series of experiments run by Bailenson suggest that when people experience racism in the form of a minority avatar, their implicit racial bias is reduced! Thus, VR can be used as a tool for helping us understand discrimination, prejudice, and racism. This concept reaches back to one of the themes for this paper which is neuroplaticity, what we see or do in VR can have an effect on how we behave in reality.
“We’ve known from our past research, when you occupy a human avatar that you gain empathy towards that human. So you can reduce racism, ageism, sexism,” — Jeremy Bailenson
Other studies with VR have shown that inhabiting the avatar of an elderly person makes subjects willing to save more for their retirement (Hershfield et al., 2011). Inhabiting the body of a person of a different color shows that they perform better on tests for implicit racial bias (Peck et al., 2013). Inhabiting the body of logger, brutally chopping down an inviting glade of trees with a chainsaw, makes people behave in a more environmentally conscious manner (Ahn et al., 2014). And even embodying the form of a cow can help subjects imagine the animal’s suffering and sacrifice when journeying from pasture to slaughterhouse.
There is even a mandatory simulation at the University of New England that medical students experience in order to understand the experience of dying.
Disclaimer: I am not suggesting that VR designers force behavior modification into their applications. If individuals do not seek to alter their psychological profile in the ways intended by the beneficent VR interventions, then such interventions may be considered a violation of their autonomy. This is where honesty in design becomes a must.
If I think of where America is now today, we have a some major social movements that VR for Empathy can play a role in. One of these is the Me Too movement, and the other is the Black Lives Matter movement. At this point, I’ll leave it up to the reader to imagine how VR can play a role to produce empathy for the people suffering on the other side of these movements.
Education
“Right now our culture is just at the very beginning of shift from the Information Age to the Experiential Age where our attention will be primarily focused on experiencing visceral emotions rather than consuming vast data streams of information.” — Kent Bye, Voices of VR
I’ve spent a lot of time listening to Kent Bye’s work on the Voice’s of VR PodCast. For those who are interested in the field, keeping up with his PodCast is an absolute must. Every week or so there is a new 30–60+ minute interview with someone in the VR industry working on something in VR that is helping to push the field forward. One thing I’ve recognized throughout listening to his PodCast is that VR is engaging- and I wouldn’t be the first to say that student engagement is a hallmark of great education.
Susan Jang, Jonathan Vitale, Robert Jyung and John Black coverered a few reasons why VR is such a powerful tool for education in a 2017 review. Some examples include:
- VR systems allow users to explore immersive, three-dimensional (3-D) environments from any location, which could have a profound impact on science education (Merchant, Goetz, Cifuentes, Keeney-Kennicutt, & Davis, 2014).
- VR affords investigation of distant locations, exploration of hidden phenomena, and manipulation of otherwise immutable structures (Lee & Wong, 2014).
- VR can help medical students explore delicate internal organs that would otherwise require cadaver dissection (Nicholson, Chalk, Funnell, & Daniel, 2006).
Findings from Jang et al. 2016 suggest that participants in a manipulation group were more likely to successfully draw anatomical structures at post-test than a viewing (control) group. The results suggest that direct manipulation of objects inside of VR facilitated embodiment of the anatomical structure, and helped participants maintain a clear frame of reference while interacting.
The video below shows how VR can be used to help better understand the human body. Warning: contents are graphic
Education is an example of where the distracting nature of VR can be used for good. It can isolate students or doctors from potential distractions in the real world and focus their attention on the simulation. Furthermore, education in VR allows us to do things that are dangerous, impossible, or expensive without the risk of spending resources. To put it bluntly, VR allows us to experiment with as little price to pay as possible.
“Thirty-fifth VR Definition: Training simulators for anything. Not just flight.” -Jaron Lanier
The Chinese Ministry of Education recently partnered with Microsoft China to start introducing VR to the classroom. According to the plan, the Management Information Center will build VR laboratories in selected schools nationwide. Microsoft China will provide necessary resources in VR software and hardware, including training courseware, teacher training, and Microsoft certification.
Walmart recently announced that they will also be introducing 16,000 Oculus Go VR headsets to their workforce for training purposes. “Walmart was one of the first companies to benefit from VR’s ability to enrich employee education, and its applications will only grow from here,” Oculus business partnerships head Andy Mathis said in a press release. “What makes it so compelling is that costly, difficult, or otherwise impossible scenarios and simulations become not only possible, but immediately within reach.”
United Postal Service (UPS) also recently announced that they will use VR headsets to train student drivers to avoid road obstacles. UPS notes that VR provides student drivers with a realistic training environment. It represents the experience of driving a real package delivery truck. Those in training will wear headsets like the HTC Vive that will show a simulation of the road and hazards. The simulations were created in-house and the training modules will test students in identifying pedestrians, parked cars, and oncoming traffic. This is an example of VR being used to prevent accidents in the real world which is a very human-centered approach to VR design.
This is only the beginning for how VR will impact education. I imagine that once the accessibility of VR headsets is improved, where powerful headsets are just a fraction of their current costs, then even those who would never be able to receive a decent education will be able to attend a virtual university and learn about whatever they would like to. And who knows, maybe one day they could make money through VR as well. After all, this is how the protagonist of Ernest Cline’s “Ready Player One” attended school.
Exercise
A recent study from the University of Kent in the UK found that VR might be able to help people train longer and harder. This can be attributed to the elements of VR that make it a tool for distraction. In the case of exercise, this is distraction from muscle fatigue and pain. Overall, participants who wore VR headsets mid-workout had a lower heart rate, lower reported pain intensity, and lower perceived exertion compared to the control group. The participants were also able to exercise up to three minutes longer than the control group.
Games like Beat Saber, Audio Shield, and Box VR are examples of VR software that make exercise fun. According to the VR Institute of Health and Exercise, over 152,000,000 calories have been burnt by playing the game Audio Shield.
As an avid kickboxer, I am used to burning upwards of 1000 calories per a one hour session. I spent a month training using Box VR and found myself burning upwards of 500 calories for a half hour session. It was amazing how powerful of an effect the gamification of exercise had on me. I seriously enjoyed racking up combos and throwing strong upper-cuts at screaming neon skulls to the rhythm of electronic dance music. I discovered I was not alone. A redditor going by the handle BigRob7605 recently posted about losing over 138 pounds by playing the game Beat Saber over the course of three months!
Part of what is in play here is something called the flow state. It is a similar state that creatives refer to when getting lost in a creative trance, losing track of time and distractions. It’s that magical moment where they become fully immersed and “one” with the article, art piece, or new design they’re working on. Achieving a flow state is what allows people to accomplish incredible things in a seemingly short period of time and is likened to meditation. Combine that with music and a gamified experience, and flow states can lead to exceptionally powerful results.
If you are interested in learning more about fitness, flow states, and exercise in VR, check out https://www.vrfitnessinsider.com/. It is a publication dedicated entirely to VR Fitness.
Conclusion — The Showdown
“ When social networking feels like the best party you’ve ever been to, when online gambling feels like going to Vegas, when pornography feels like sex, how does society function?” — Jeremy Bailenson
This paper was created for anyone interested in VR- whether you’re a fellow designer, a developer, a business owner, or just someone who wants to learn more about the subject. My hope is that this paper showed how VR could be a utopia or a dystopia, and gave you some ideas that lead more to the former.
VR requires empathy, emotion, care, and imagination. Most of all, it requires trust. Trust that simulations will not deceive us or make us feel manipulated. In order for the technology to succeed, people will need to trust surrendering their perception to technology. This is a vulnerable feeling- ask anyone trying VR for the first time. A single bad experience with the technology can make someone lose trust in it forever, we need to earn it.
VR has the potential to be an awesome invention, giving people the opportunity to generate just about any experience. The technology has a long way to go until it reaches the expectations of cyberspace visionaries, but we are well on our way. The bulky headsets we wear today will most likely be obsolete once these visions are met- being replaced by headbands, glasses, contacts, full haptic suits, or even neural implants. Despite the change in form, the functionality requirements for VR will likely stay the same.
Once VR reaches a certain fidelity, we could very well have a hard time distinguishing virtual reality from reality all together. Early experiments in prolonged immersion already show that this phenomena can take place. In this regard, VR also has the potential to be one of the most psychologically manipulative inventions ever created. It can control, change and violate people’s autonomy.
We live in a reality where it is already difficult to separate the real from unreal, where corporations are abusing technologies influence for gain, and people are getting hurt because of poor and careless designs. We are seeing technology be abused in a way that uses people’s data against them, and addicts people and takes advantage of their human instincts.
We now have the opportunity to step up and make VR a technology for good. This is a call to action for designers, developers and business leaders to work together and think about the methods that will create virtual experiences for the greater good. Something that can assist our reality without replacing it entirely. Perhaps even make it a little better.
Let’s make VR the greatest technology there has ever been.
This is only the beginning.
Special Thanks
A big thank-you to my close friend Brian Brennan for helping me edit this piece. A special thanks to Balance Innovation and Design for supporting my interest in VR. Thank you to my family. Finally, thank you to the reader for your interest in my writing.
Sources
Publications
Ahn, S. J., Bailenson, J., and Park, D. (2014). Short- and long-term effects of embodied experiences in immersive virtual environments on environmental locus of control and behavior. Comput. Human Behav. 39, 235–245. doi:10.1016/j. chb.2014.07.025
Albiol-Pérez, S., Forcano-Garcia, M., Munoz-Tomas, M.T., Manzano-Fernandez, P., Solsona-Hernandez, S., Mashat, M.A., Gil-Gomez, J.A., 2015. A novel virtual motor rehabilitation system for guillain-barré syndrome. Meth. Inf. Med. 54 (2), 127–134.
Allcoat, Devon, et al. “Frozen with Fear: Conditioned Suppression in a Virtual Reality Model of Human Anxiety.” Behavioural Processes, vol. 118, 2015, pp. 98–101., doi:10.1016/j.beproc.2015.06.011.
Anderson-Hanley, C., Synder, A. L., Nimon, J. P., & Arciero, P. J. (2011). Social facilitation in virtual reality-enhanced exercise: competitiveness moderates exercise effort of older adults. Clinical Interventions in Aging, 6,275e280.
Annerstedt, Matilda, et al. “Inducing Physiological Stress Recovery with Sounds of Nature in a Virtual Reality Forest — Results from a Pilot Study.” Physiology & Behavior, vol. 118, 2013, pp. 240–250., doi:10.1016/j.physbeh.2013.05.023.
Clark, Andy. Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. Oxford Univ. Press, 2010.
Cornwell, Brian R., et al. “Anticipation of Public Speaking in Virtual Reality Reveals a Relationship Between Trait Social Anxiety and Startle Reactivity.” Biological Psychiatry, vol. 59, no. 7, 2006, pp. 664–666., doi:10.1016/j.biopsych.2005.09.015.
Coyle, J., and Thorson, E. (2001). The effects of progressive levels of interactivity and vividness in web marketing sites. J. Advert. 30, 65–77. doi:10.1080/00913 367.2001.10673646
Cummings, J. J., & Bailenson, J. N. (2016). How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology, 19(2), 272–309.
de Jaegher, H., Di Paolo, E., and Gallagher, S. (2010). Can social interaction constitute social cognition? Trends Cogn. Sci. 14, 441–447. doi:10.1016/j. tics.2010.06.009
Diemer, Julia, et al. “Virtual Reality Exposure in Anxiety Disorders: Impact on Psychophysiological Reactivity.” The World Journal of Biological Psychiatry, vol. 15, no. 6, 2014, pp. 427–442., doi:10.3109/15622975.2014.892632.
Fulk, G., 2005. Locomotor training and virtual reality-based balance training for an in- dividual with multiple sclerosis: a case report. J. Neurol. Phys. Ther. 29 (1), 34–42.
Fredrickson, B. L., & Branigan, C. (2005). Positive emotions broaden the scope of attention and thought-action repertoires. Cognition and Emotion, 19, 313–332.
Frith, C., and Frith, U. (2007). Social cognition in humans. Curr. Biol. 17, R724–R732. doi:10.1016/j.cub.2007.05.068
Gallagher, S. (2008). Inference or interaction: social cognition without precursors. Phil. Explor. 11, 163–174. doi:10.1080/13869790802239227
Gregg, L., and Tarrier, N. (2007). Virtual reality in mental health. A review of the literature. Soc. Psychiatry Psychiatr. Epidemiol. 42, 343–354. doi:10.1007/ s00127–007–0173–4
Gromala, Diane, et al. “Mobius Floe: an Immersive Virtual Reality Game for Pain Distraction.” Electronic Imaging, vol. 2016, no. 4, 2016, pp. 1–5., doi:10.2352/issn.2470–1173.2016.4.ervr-413.
Haik, J., Tessone, A., Nota, A., Mendes, D., Raz, L., Goldan, O., Regev, E., Winkler, E., Mor, E., Orenstein, A., Hollombe, I., 2006. The use of video capture virtual reality in burn rehabilitation: the possibilities. J. Burn Care Res. 27 (2), 195–197.
Haslet, Béatrice S., et al. “Virtual Race Transformation Reverses Racial in-Group Bias.” Plos One, vol. 12, no. 4, 2017, doi:10.1371/journal.pone.0174965.
Hershfield, H., Goldstein, D., Sharpe, W., Fox, J., Yeykelis, L., Carstensen, L., et al. (2011). Increasing saving behavior through age-progressed renderings of the future self. J. Mark. Res. 48, S23–S37. doi:10.1509/jmkr.48.SPL.S23
Hodges, L., Anderson, P., Burdea, G. C., Hoffman, H. G., & Rothbaum, B. O. (2011). Treating psychological and physical disorders with VR. IEEE Computer Graphics and Applications, 21(6), 25–33.
Hoffman, H.G., Garcia-Palacios, A., Patterson, D.R., Jensen, M., Furness, T., and Ammons, W.F. The effectiveness of virtual reality for dental pain control: a case study. Cyberpsychology & Behavior: The Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society 4, 4 (2001), 527–535.
Jack, D., Boian, R., Merians, A.S., Tremaine, M., Burdea, G.C., Adamovich, S.V., … Poizner, H., 2001. Virtual reality-enhanced stroke rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 9 (3), 308–318.
Jang, Susan, et al. “Direct Manipulation Is Better than Passive Viewing for Learning Anatomy in a Three-Dimensional Virtual Reality Environment.” Computers & Education, vol. 106, 2017, pp. 150–165., doi:10.1016/j.compedu.2016.12.009.
Kilteni, K., Bergstrom, I., and Slater, M. (2013). Drumming in immersive virtual reality: the body shapes the way we play. IEEE Trans. Vis. Comput. Graph. 19, 597–605. doi:10.1109/TVCG.2013.29
Leary, M. R., & Kowalski, R. M. (1995). Social anxiety. New York: Guilford.
Lee, E. A. L., & Wong, K. W. (2014). Learning with desktop virtual reality: Low spatial ability learners are more positively affected. Computers and Education, 79,49e58.
Levin, A. P., Saoud, J. B., Strauman, T., Gorman, J. M., Fryer, A. J., Crawford, R., et al. (1993). Responses of “generalized” and “dis- crete” social phobics during public speaking. Journal of Anxiety Disorders, 7, 207–221.
Madary, M. (2014). Intentionality and virtual objects: the case of Qiu Chengwei’s dragon sabre. Ethics Inf. Technol. 16, 219–225. doi:10.1007/s10676–014–9347–4
Madary, Michael, and Thomas K. Metzinger. “Recommendations for Good Scientific Practice and the Consumers of VR-Technology.” Frontiers in Robotics and AI, vol. 3, 2016, doi:10.3389/frobt.2016.00003.
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers and Education, 70,29e40.
Metzinger, T., and Hildt, E. (2011). “Cognitive enhancement,” in The Oxford Handbook of Neuroethics, eds J. Illes and B. J. Sahakian (Oxford, NY: Oxford University Press (Oxford Library of Psychology)), 245–264.
Mirelman, A., Maidan, I., Herman, T., Deutsch, J., Giladi, N., Hausdorff, J., 2010. Virtual reality for gait training: can it induce motor learning to enhance complex walking and reduce fall risk in patients with Parkinson’s disease? Journals of Gerontology Series A: Biomedical Sciences and Medical Sciences 66 (2), 234–240.
Mühlberger, Andreas, et al. “Virtual Reality for the Psychophysiological Assessment of Phobic Fear: Responses during Virtual Tunnel Driving.” Psychological Assessment, vol. 19, no. 3, 2007, pp. 340–346., doi:10.1037/1040–3590.19.3.340.
Mühlberger, A., Petrusek, S., Herrmann, M. J., & Pauli, P. (2005). Biocyberpsychologie: Subjektive und physiologische Reaktionen von Flugphobikern und Gesunden bei Exposition mit virtuellen Flügen. [Biocyberpsychology: Subjective and physiological reactions in flight phobics and normal subjects during flight simulations]. Zeitschrift für Klinische Psychologie und Psychotherapie: Forschung und Praxis, 34(2), 133–143. http:// dx.doi.org/10.1026/1616–3443.34.2.133.
Nicholson, D. T., Chalk, C., Funnell, W. R. J., & Daniel, S. J. (2006). Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Medical Education, 40(11), 1081e1087
Nunes, M., Nedel, L., & Roesler, V. (2014). Motivating people to perform better in exergames: collaboration vs. competition in virtual environments. In Proceedings of the 29th annual ACM symposium on applied computing (pp. 970e975). New York: ACM Press.
Opris, D., Pintea, S., Garcia-Palacios, A., Botella, C., Szamoskozi, S., & David, D. (2012). Virtual reality exposure therapy in anxiety disorders: A quantitative meta- analysis.
Depression and Anxiety, 29(2), 85–93. http://dx.doi.org/ 10.1002/da.20910.
Paulus, D., and Williams, K. (2002). The dark triad of personality. Narcissism, Machiavellianism, and psychopathy. J. Res. Pers. 36, 556–563. doi:10.1016/ S0092–6566(02)00505–6
Peck, T., Seinfeld, S., Aglioti, S., and Slater, M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious. Cogn. 22, 779–787. doi:10.1016/j.concog.2013.04.016
Reid, D.T., 2002. Benefits of a virtual play rehabilitation environment for children with cerebral palsy on perceptions of self-efficacy: a pilot study. Pediatr. Rehabil. 5 (3), 141–148.
Riva, Giuseppe, et al. “Affective Interactions Using Virtual Reality: The Link between Presence and Emotions.” CyberPsychology & Behavior, vol. 10, no. 1, 2007, pp. 45–56., doi:10.1089/cpb.2006.9993.
Riva, G., Waterworth, J., Waterworth, E., and Mantovani, F. (2011). From intention to action: the role of presence. New Ideas Psychol. 29, 24–37. doi:10.1016/j. newideapsych.2009.11.002
Rose, Tyler, et al. “Immersion of Virtual Reality for Rehabilitation — Review.” Applied Ergonomics, vol. 69, 2018, pp. 153–161., doi:10.1016/j.apergo.2018.01.009.
Shriram, Ketaki, and Raz Schwartz. “All Are Welcome: Using VR Ethnography to Explore Harassment Behavior in Immersive Social Virtual Reality.” 2017 IEEE Virtual Reality (VR), 2017, doi:10.1109/vr.2017.7892258.
Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (five): Speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environmesnts, 6, 603-616.
Slater, M., Antley, A., Davison, A., Swapp, D., Guger, C., Barker, C., et al. (2006). A virtual reprise of the Stanley Milgram obedience experiments. PLoS ONE 1:e39. doi:10.1371/journal.pone.0000039
Snyder, A. L., Anderson-Hanley, C., & Arciero, P. J. (2012). Virtual and live social facilitation while exergaming: competitiveness moderates exercise intensity. Journal of Sport & Exercise Psychology, 34, 252e259.
Sveistrup, H., 2004. Motor rehabilitation using virtual reality. J. NeuroEng. Rehabil. 1 (1), 10.
Teo, A., Choi, H., Andrea, S., Valenstein, M., Newsom, J., Dobscha, S., et al. (2015). Does mode of contact with different types of social relationships predict depres- sion in older adults? Evidence from a nationally representative survey. J. Am. Geriatr. Soc. 63, 2014–2022. doi:10.1111/jgs.13667
Tussyadiah, Iis P., et al. “Virtual Reality, Presence, and Attitude Change: Empirical Evidence from Tourism.” Tourism Management, vol. 66, 2018, pp. 140–154., doi:10.1016/j.tourman.2017.12.003.
Wang, Guang-Xin, and Li Li. “Virtual Reality Exposure Therapy of Anxiety Disorders.” Advances in Psychological Science, vol. 20, no. 8, 2013, pp. 1277–1286., doi:10.3724/sp.j.1042.2012.01277.
Williams, Kipling D., et al. “Cyberostracism: Effects of Being Ignored over the Internet.” Journal of Personality and Social Psychology, vol. 79, no. 5, 2000, pp. 748–762., doi:10.1037//0022–3514.79.5.748.
Williams, Kipling, and Jonathan Gerber. “Ostracism: The Making of the Ignored and Excluded Mind.” Interaction Studies Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems Making Minds II, vol. 6, no. 3, 2005, pp. 359–374., doi:10.1075/is.6.3.04wil.
Yee, N., and Bailenson, J. (2007). The proteus effect. The effect of transformed self-representation on behavior. Hum. Commun. Res. 33, 271–290. doi:10.1111/j.1468–2958.2007.00299.x
Books
Websites
Image effects were generating using the free online tool: Photomosh
