Learning and Embodiment in the Metaverse: a Literature Review

I bought my first virtual reality headset when I was 17 years old, it was one of the first models and it made me incredibly motion sick but I still loved it. I eventually sold it because I grew tired of the thing and didn’t see much of a future for it in my life. Fast forward 5 years later. I was recently playing around with one of the newest VR headsets at my school. I started up a VR demo called “The People’s House”, it was a virtual reality tour of the White House. I came to on the lawn of the White House and former President Obama’s voice was narrating in my ear. We moved through different rooms and he explained what living in the White House meant to him and the first lady. Then we shifted to the Oval Office and for the first time his voice sounded like it was coming from my left. I turned my head and there he was in full life size 3D looking right into my eyes. I watched in amazement as he continued to talk to me. I was overcome by a more powerful emotional response than I am happy to admit. It really felt like he was there, talking to me, in that moment I really understood the power of real virtual realty. The virtual reality market today is dominated by entertainment and gaming but I hope to prove that this platform can be used to teach us about ourselves, the people around us, and our place on this planet.

The history of virtual reality is a story paved with failures. One of the earliest attempts at an alternate reality was the Sensorama. It was a mechanical device that could display stereoscopic 3-D images in wide-angle, tilt the viewers body, provide stereo sound, and also could blow wind and aromas. (Sensorama, US Patent Office). However Morton Heilig, the inventor was never able to get investors so it never took off. Virtual reality didn’t get it’s first true realization until a man named Ivan Sutherland built the first computerized virtual reality headset in the 1960s. The device allowed people to view into a 3D wireframe world and it was truly revolutionary. The headset, nicknamed the Sword of Damocles, was so heavy it needed dentist machinery like arms to hold up the headset. Sutherland wrote a paper titled “The Ultimate Display” where he spoke of the power of virtual worlds as a tool of education. He remarked that humans are incredibly good at understanding the physical forces in the world we are born into, like gravity or the perceived weight of objects. However, Sutherland then noted how we lack the same sense about the invisible world he said “We lack corresponding familiarity with the forces on charged particles, forces in non-uniform fields, the effects of nonprojective geometric transformations, and high-inertia, low friction motion. A display connected to a digital computer gives us a chance to gain familiarity with concepts not realizable in the physical world. It is a looking glass into a mathematical wonderland”(Sutherland, 1965). He aimed to use virtual reality as a window into this invisible world to teach a new generation of students. While Sutherland’s technology was extremely innovative for its era the market had demonstrated its interest in 2D flat displays instead. With the improving computer graphics of the 1980s, research on virtual reality continued in the background. The first consumer virtual reality headset was the Virtual Boy from Nintendo, released in August 1995. The device is now regarded as a complete failure. While marketed heavily by Nintendo, the device was mono-chromatic, had few games, and lacked head tracking which caused its users to get sick immediately (Boyer, 2009). The Virtual Boy was pulled from the market the following year. It was clear that virtual reality wasn’t ready for the prime time. The consumer virtual reality market seemed all but dead until 2012.

On August 1 2012 the Oculus Rift VR headset was posted on Kickstarter thus starting the revival of the virtual reality industry. The Oculus Rift went on to raise a total of $2.5 million and was eventually bought for 2 billion by Facebook. So virtual reality made a comeback, but what makes now so different than the time that has come before? There are a lot of factors at play that make right now plenty different from the past. To start, the Oculus kickstarter essential started an arms race for virtual reality. Facebook, HTC, Sony, Microsoft, and many more all have begun to develop their own technology. This vastly contrast the market space of the 1990s where there were only a few companies brave enough to dabble in VR. This leads to the second point, the technology has improved greatly, with all of these companies trying to out sell each other the technology will only improve with time. The headsets of today are smaller and cheaper than before. The headsets use high quality displays with full 3d, combined with zero latency head and full body tracking provides for a much more immersive and believable experience compared to the late 90s. The renewed interest were seeing in virtual reality in 2017 has also re-sparked the publics interest in augmented realty as well. Digi-Capital, a market research firm is projecting that by 2021 the AR/VR market will be 108 billion with AR taking most of that pie (Digi-Capital, 2009).

Apple’s CEO Tim Cook recently pledged his allegiance to AR by saying “I regard it as a big idea, like the smartphone. The smartphone is for everyone, we don’t have to think the iPhone is about a certain demographic, or country or vertical market: it’s for everyone. I think AR is that big, it’s huge”.

As of this September Apple just released iOS 11 which enabled 100s of millions of iOS devices to become augmented reality ready. Palmer Lucky, the orignial founder of the Oculus rift was recently speaking at a conference when he said

“That VR was selling more headsets than compared the iPod at the same point in its life”.

It’s clear that this wave is not like the ones that have come before it, the excitement around VR/AR is real and its here to stay. At my school, California College of the Arts, we recently had a designer from the Facebook VR team come speak. He compared the current landscape to the wild west or the gold rush. “It’s any mans game”. He likened it to the pinch to zoom era post iPhone, everyone was wracking their brain like why didn’t I think of that. He told us “you could be the designer to come up with pinch to zoom for virtual reality”. VR/AR has the potential to be world changing just like the smartphone but I see a problem with the current landscape. Almost everyone developing for virtual worlds right now is developing some sort of game or entertainment experience, while important there are other uses that should be explored with the ferocity that games are being explored. I get excited thinking about all the uses for VR/AR outside of gaming, one such area is education.

Forget about virtual worlds for a second. I want to talk about embodied learning and embodied cognition. These are the theories that say many features of cognition are shaped and directly tied to aspects of the entire body. I recently read a terrific anecdote for embodied learning “Picture this. A preverbal infant straddles the center of a seesaw. She gently tilts her weight back and forth from one side to the other, sensing as each side tips downward and then back up again. This child cannot articulate her observations in simple words, let alone in scientific jargon. Can she learn anything from this experience? If so, what is she learning, and what role might such learning play in her future interactions in the world?”(Abrahamson & Lindgren). I thought this was such a beautiful way to picture embodied learning. As human beings we learn to move through space far before we learn to reason with words and symbols. The scientific community is still exploring the extent to which our minds are intractable from our bodies. One study showed that when we hear the verbs “Lick,” “ Pick,” and “Kick,” our brain secretly activates the parts of the motor system that correlate to each verb, like the mouth, hands and legs (Hauk, Johnsrude, & Pulvermuller, 2004). Another central study showed that students who used gestures whilst leaning math retained the information better than those who didn’t (Cook & Mitchell, 2009). Developmental psychologists broadly agree that bodily action plays a central roll in conceptual development.

So what does all of this have to do with virtual or augmented reality. I’m arguing that virtual and augmented reality are the perfect platforms for the next wave of education tools. Science is just recognizing the fact that humans learn first and foremost with their bodies. I propose that we rethink how we learn and conceptualize the world around us with the use of these new digital tools. When you hear the word education it’s easy to think about it in the traditional sense, like K-12 or higher education. However, I am choosing to take the word in a much broader sense. Education is learning and I believe that AR/VR has the potential to change the way all humans learn and relate to the world around them at all stages of life from birth to death. Virtual and augmented reality have the power to change the landscape of education over the next decade in a powerful way (Lindgren & Glenberg, 2017).

I am not sure what area of education I would like to focus on but I have narrowed it down three buckets; how we learn about ourselves, how we learn about the world around us, and how we learn about our place in the universe. The first is internal, how can 3D digital augmented space provide a place for us to keep store and learn from past memories. The next is external, how can we use this platform to learn about the history of those who have come before and the world around us currently. Finally, the third is a beautiful blend of the previous two how can we use 3D digital space to teach humans about their natural place in the universe and how to best live in it.

Steve Jobs famously remarked that the computer was like a bicycle for the mind. This has been more than true in my time growing up. I am probably one of the last generations of that will have had to remember a phone number by memory. There have been three digital booms before me; PC revolution, Internet revolution, and the mobile revolution. I can remember the end of the last boom and thats exactly what drove me to become a passionate User Experience designer. The fourth wave is just starting to break and it is Virtual and Augmented reality. I want to help make a contribution to the next medium of human expression and learning.

Works Cited:

1. Sensorama simulator.” U.S. Patent 3,050,870, issued August 28, 1962.
2. Sutherland, Ivan E. “The ultimate display.” Multimedia: From Wagner to virtual reality (1965).
3. Boyer, Steven. “A virtual failure: Evaluating the success of Nintendo’s Virtual Boy.” The Velvet Light Trap 64 (2009): 23–33.
4. After mixed year, mobile AR to drive $108 billion VR/AR market by 2021, https://www.digi-capital.com/news/2017/01/after-mixed-year-mobile-ar-to-drive-108-billion-vrar-market-by-2021/
5. Abrahamson, Dor, and Robb Lindgren. “Embodiment and embodied design.” The Cambridge handbook of the learning sciences 2 (2014): 358–376.
6. Hauk, Olaf, Ingrid Johnsrude, and Friedemann Pulvermüller. “Somatotopic representation of action words in human motor and premotor cortex.” Neuron 41.2 (2004): 301–307.
7. Cook, Susan Wagner, Zachary Mitchell, and Susan Goldin-Meadow. “Gesturing makes learning last.” Cognition 106.2 (2008): 1047–1058.
8. Lindgren, Robb, and Mina Johnson-Glenberg. “Emboldened by embodiment: Six precepts for research on embodied learning and mixed reality.” Educational Researcher 42.8 (2013): 445–452.