Extended Reality
The Potential of Augmented, Virtual, and Mixed Reality Experiences for Remote Teaching and Learning
By Melyssa Eigen, Sandra Cortesi, & Alexa Hasse
In collaboration with Pedro Maddens Toscano, Maya Malik, Leah Plunkett, & Urs Gasser (PI). Illustrations and animations by Euan Brown, Rebecca Smith, Melanie Tan, & Claudia Thomas
Due to the global pandemic, the past several months have been a whirlwind of changing norms and lifestyle modifications, as well as an increase in impact of the digitalization of our day-to-day lives (highlighting the importance of high-quality access to digital technologies and areas where such access might be improved). With the inability for life to “return to normal” anytime soon, we are bound to see more change, especially in the educational context. For some of us, remote teaching is the only option to continue school. There are some guidance documents, but each state, school district, and individual educator has been asked to assume significant leadership in shaping their own remote learning space and defining the technologies available towards creating a successful online experience.
For educators and students with high-quality access to digital technologies and the skills and agency to use these tools and platforms (the pandemic has shed new light on the persistence of digital divides and participation gaps in the learning space and the urgency to close these gaps), there are a number of opportunities available to enhance their remote teaching and learning experiences to help them in their transition. The tools range from learning platforms such as KhanAcademy; social media platforms like Facebook; video sites such as YouTube; storing and sharing platforms like Google Classroom or Dropbox; and more playful platforms like Minecraft and Scratch, or our own Digital Citizenship+ (Plus) Resource Platform (DCPR), that hosts an evolving collection of free and Creative Commons-licensed learning experiences, visualizations, and other educational resources (many of these resources are available in over 35 languages).
The XR environment includes multiple technologies, like augmented reality (AR), virtual reality (VR), and mixed reality (MR). It allows users to integrate a variety of creative technologies in order to expand both their real-world and virtual surroundings. With AR, users can overlay virtual effects in their real-world environment, whereas with VR, users can act in a completely virtual environment. Further, MR gives users the ability to use real-world objects when interacting in virtual environments. All in all, there are a lot of possibilities for users under the “umbrella” of XR technologies. In some instances, users may need VR headsets, but in other cases, individuals can access the technologies with just a mobile device or computer and access to the Internet.
One emerging set of tools in this space falls under the category of extended reality technology (XR), which is becoming increasingly prevalent in the lives of young people. For example, as part of an ongoing project by Youth and Media at the Berkman Klein Center for Internet & Society at Harvard University, we are exploring the potential roles XR may play in the youth context while also recognizing the various access and usage barriers. A forthcoming publication — sign up for our team’s mailing list here to receive an early copy — addresses XR as it relates to domains such as education, physical and mental health, and social issues, as well as the potential roadblocks that stand in the way of these technologies becoming widely available. Additionally, the publication introduces the reader to over 30 XR experiences that have the potential to enhance teaching and learning across these domains in the youth context.
There are a variety of XR tools that can be explored in remote environments that may help improve the teaching and learning experience, enhance students’ motivation and engagement, or simply make a learning activity more novel or fun. It’s important to note, however, that while there is an increasing evidence base around using AR and VR to teach different subjects, a great deal of the literature focuses on using these technologies to teach a certain course or topic area. As the following report indicates, however, far fewer studies examine the efficacy of XR technologies compared to the use of non-XR tools in the educational space. As such, we envision XR technologies as one possible tool educators might consider integrating in the classroom — depending upon the affordability and accessibility of these technologies, which we elaborate upon further in this piece.
Preliminary evidence indicates three key areas with respect to potential ways XR technologies may be integrated in the classroom. First, XR technologies can be used to foster skill-based learning, such as learning a language. Research demonstrates that immersion may be helpful in learning a second language, and that VR, for example, can effectively simulate an immersive language experience. Second, XR can expand the possible activities youth can learn from in a hands-on manner. Such technologies can, for example, allow young people to travel inside the human body and explore cells for a biology course, or, for a physics class, examine how charged particles can interact with each other. Thus, XR has the potential to increase the topics young people can learn from by turning abstract concepts into concrete experiences. And third, XR can allow for new functionalities, or affordances, that allow for young people to learn in ways that have not yet been possible with other technological tools. For students taking vocational classes interested in architecture or construction, for instance, XR can simulate architectural designs that are more realistic that computer-aided designs — allowing individuals to walk through a space and explore various objects in it.
In addition to these three areas, there are several other areas connected to the educational space that appear promising in the context of XR. For example, XR technologies may be able to help fill in some of the gaps created by transitioning from in-person to remote learning. Against the backdrop of the global pandemic and disrupted education in regions around the world, field trips, which are often one of the most exciting aspects of formal education, are unlikely to occur in the coming months. While not equivalent to an in-person experience, XR technologies, such as VR, can create the opportunity to explore places that have not been possible for many to travel to. For example, students could explore the depths of the ocean or even take a rocket to outer space. Instead of in-person field trips to museums, students can use XR technologies to explore exhibits — some museums have their own XR-based apps, such as the Louvre’s “Mona Lisa: Beyond the Glass” VR app or the “Anne Frank House VR” experience. Additionally, some XR technologies allow young people to create their own AR and VR exploration experiences.
XR technologies have also been designed with the aim to cultivate skills that are relevant in (and out) of the classroom, both for educators and students. For example, there are XR experiences that allow educators to practice their pedagogical skills and those that seek to strengthen students’ presentation skills. In addition to skill-building, XR technologies can simulate different scenarios to prepare educators and students alike for emergency situations. For example, in a remote health class, one could use an XR app to teach students CPR. Other use cases could involve different forms of disaster prevention as well as vocational training.
Another promising application of XR technology centers around addressing social issues in the educational setting, such as how to prevent and respond to bullying. In the context of remote learning, while students may not be in physical contact with one another, this does not eliminate the potential for bullying. Given the isolated nature of remote learning, anti-bullying programming remains vital to the educational setting. In response, there are a number of different XR technologies that help students learn about one another’s differences in an effort to prevent and mitigate bullying, which educators may implement in a remote curriculum. These platforms use a variety of techniques to promote empathy, ranging from stand-up comedy, to placing the user in the role of the bully or bullied, to focusing on training teachers to identify bullying.
With respect to the adoption of XR technologies in the educational space, research in the context of higher education suggests two key factors are important. First, the XR technologies need to fit into current curriculum standards and educators’ instructional methods. This finding aligns with research around the diffusion of innovations more broadly. For any innovation to be implemented in a given setting — from the workplace to the classroom — it must fit within systems currently in place. Second, as with the diffusion of other innovations, the adoption of XR is influenced by the cost of these technologies — not only in terms of the monetary value, but, for instance, the cognitive load of learning how to use XR.
While this brief overview focuses on the opportunities that XR technologies may present in the educational space, it’s also important to highlight that these technologies come with challenges related to accessibility, privacy, and safety. In order to take advantage of the affordances of XR, as with other digitally networked technologies, Internet connectivity is needed. This may prevent a significant number of people from leveraging the technology — even more so during the pandemic with many schools and libraries closed. Further, we often associate XR with expensive VR headsets, which can be financially burdensome and inaccessible to many. These barriers are real. At the same time, less costly VR headset options, such as Google Cardboard, as well as free and low-cost VR and AR apps have become available, which students can use on their phones without further equipment.
Additionally, as with many other networked technologies, such as artificial intelligence, there are concerns around the extent to which the design of XR incorporates the voices and perspectives of underrepresented groups — whether in terms of age, ethnicity, race, gender and sexual identity, religion, national origin, location, skill and educational level, and/or socioeconomic status. In terms of additional issues around inclusion, XR is more reliant (compared to other digital technologies) on individuals’ ability to control their physical motions (e.g., quickly moving one’s hand). How can these technologies be made more accessible for those with limited mobility?
Second, there are numerous challenges around privacy, data and data protection, and commercial risk, some of which are resonant with general concerns that come with digitally networked technologies, while others are specific to XR. For example — like many digitally connected systems — XR has the ability to collect, aggregate, analyze and monetize users’ data — data which is “durable, searchable, and virtually undeletable.” Given that children and youth are often pioneers in exploring emerging technologies, they may experience XR-related privacy and data protection risks before adults enact strategies to mitigate such concerns. Moreover, the unprecedented amount of data that networked technologies such as XR are able to collect can be sold to third parties, including companies that can target marketing to youth. These targeted messages may put youth at risk for commercial exploration and exposure to content that may impact their perspectives and behaviors in ways not optimal for healthy development.
Moreover, research shows that young people (like many adults) generally lack an adequate understanding of the processes of data collection and resultant commercial profiling and marketing to which they can be subject. The risks associated with commercial profiling are only worsened in the context of COVID, with young people spending increased amounts of time in the digital environment and commerce currently being driven online. In addition to collecting information such as young people’s product preferences and location, XR technologies also have the potential to gather large amounts of data about nonverbal behavior, such as gestures, facial expressions, and eye gaze — even if one is only using the system for short periods of time. Indeed, spending a mere 20 minutes within a VR simulation may capture slightly under two million recordings of one’s body language. In the context of education, researchers have used nonverbal data gathered through VR to predict test scores, and estimate the number of mistakes made while learning a specific task. A young person’s future — from the university they are admitted to, to their employment opportunities and quality of working life — could be (positively or negatively) impacted by the nonverbal data that XR technologies collect. The data captured through XR will also strengthen companies’ efforts to target advertising to youth — a study shows that data about one’s head movements in VR is associated with how positively someone rates the content in a simulation. Legal protections against these and related privacy risks vary across jurisdictions and application areas. When it comes to student privacy issues in the context of formal education in the U.S., this publication (pages 4–5) by authors from the Youth and Media team and the Cyberlaw Clinic at the Berkman Klein Center for Internet & Society at Harvard offers both a general point of entry into the privacy analysis, as well as a roadmap for exploring additional information and engaging with decision-makers who might be involved in determining whether, how, with which students, and for what purposes XR technologies can be employed, and what privacy safeguards need to be put in place.
And third, there are safety risks connected to XR technologies. As with many other networked technologies, such as computers and mobile phones, some users may experience eye strain using XR. Unlike many other Internet-enabled technologies, however, XR may induce side effects such as nausea, dizziness, seizures, and discomfort wearing the needed equipment. Recently, there have been efforts to make VR headsets, for instance, more comfortable to wear for those of different racial and ethnic backgrounds. Additionally, XR offers another online space for young people to be cyberbullied in. Research shows that harassment in XR, such as VR, may take different forms, such as environmental (e.g., throwing virtual objects) or physical (e.g., unwanted physical contact). Given the immersive nature of XR, will these types of harassment be experienced as more intense compared to bullying via text message or chat? And how can educators, parents, and other stakeholders most effectively reduce and prevent these forms of online harassment?
With time, we hope that we as researchers will be even more equipped to understand the impact the global pandemic may have on learners from all backgrounds. By all indications, however, there are several reasons to be concerned. On a global level, students — particularly those from underrepresented communities — are facing serious disruptions in accessing learning opportunities, particularly those that are online. Projects such as UNICEF’s Voices of Youth and UNESCO’s COVID-19 Education Response, which are examining some of the challenges youth are facing in the context of learning, have found that, over the coming months, it will be increasingly crucial to ensure that young people can connect learning with their needs and interests. Engaging in interest-driven learning online with a community of supportive peers and mentors — or, connected learning — can help inspire young people to be more creative and foster their ability to learn online. As with any other educational technology, the value of XR depends on many contextual factors and will require careful assessment before it is introduced in any learning environment — whether online or in-person. However, the technology comes with the promise to provide educators with additional options in terms of remote student engagement, and ways to support and complement efforts to equip youth with the skills to participate and thrive in an increasingly digitally connected economy and society.
