Part I of a series of articles from inside NRG’s think tank, sharing insight into XR (Extended Reality) and the experiences of the future.
New advancements in Extended Reality — any form of “mediated reality” be it AR, VR or other 3D formats — present significant opportunities for enhanced real-time experiences, and the advent of 5G technologies will unlock a multitude of new use cases.
That’s why researchers and strategists at National Research Group (NRG) are producing a new in-depth study, applying market-leading research methods for measuring consumers’ reactions to uncover future potential of use cases currently on the market and ideate new user experiences.
First, we’re looking to the future, conducting in-depth interviews and roundtable discussions with experts and creators across a range of perspectives to uncover emerging use cases. The second stage of research involves ethnographies with leading-edge consumers in two 5G-enabled cities, exploring what a better “day in the life” looks like with XR, before quantifying the benefits among a nationally representative sample of consumers.
The below is a glimpse at initial takeaways from the first set of our conversations with experts across LA, SF, Seattle and NY, with: Leila Amirsadeghi, XR Pioneer and Co-Creator of The Unreal Garden; Raul Carvajal, Producer, Games for Change; Cortney Harding, Founder and CEO, Friends with Holograms; Ori Inbar, Co-Founder & Managing Partner, Super Ventures and Co-Founder & Executive Producer, Augmented World Expo; Marcie Jastrow, Senior Vice President, Immersive Media, Head of the Technicolor Experience Center, Technicolor; Amy LaMeyer, Founding Partner, The WXR Fund; Tony Parisi, Global Head of AR/VR Ad Innovation at Unity Technologies; M. Pell, Envisioneer and Author, Design Lead, The Microsoft Garage; Jenny Rodenhouse, Director, Immersion Lab, Art Center; Adaora Udoji, Director, Corporate Innovation & Entrepreneurship, RLab and Martina Welkhoff, Founding Partner, The WXR Fund.
To “augment” in the truest sense is to “complement” consumers’ lived experience.
Martina Welkhoff, Founding Partner, The WXR Fund, positions the question of how XR will break through as a design question: how to leverage increasingly sophisticated tools for more intuitive interaction. Martina shares that to “augment” in the truest sense is to “complement” consumers’ lived experience, restoring more power and agency to the consumer, even if that might go against commercial interests in some instances.
When the technology does the trick best, it empowers users, making them feel more capable to accomplish what they care about. Examples of the types of commerce experiences that empower consumers include augmented visual search tools enabling consumers to be more informed shoppers through community reviews and price comparisons, or real-time translation features facilitating more meaningful communication by overcoming language barriers.
How might 5G make even more sticky experiences possible?
In the words of Tony Parisi, Head of AR/VR Ad Innovation at Unity Technologies, “5G signals richer experiences coming down the line faster. Immersive media will be one of the major media types delivered over 5G and will at some point become a motivator for having these new devices and the networks deployed.” He likens an imminent immersive content boom to the roll-out of 4G: “I don’t think people bought 4G phones to watch their HBO shows on their phones but now they’re doing that routinely. So was it a driving use case? I don’t know. Was it a lasting use case? It’s here to stay, and it’s changed behaviors worldwide.”
Tony emphasizes that the ability to create deep immersive content will also shift interactive advertising away from short-form AR try-on’s to longer form experiences, and render purpose-built apps for individual brands/experiences a thing of the past, as 5G will allow for a more seamless progression from see it to want it and buy it. He points out how already, brands are enthusiastic about new possibilities for delivering a 3D representation of their product that can be accessed in multiple ways. Immersive media and the shift from 2D to 3D can change everything from how retailers plan their stores and merchandising, to the display of the products themselves — through a touchpoint in-store, a smart display wall, or apps where AR augments the in-store experience with added layers of information.
The industry is still at its infancy in realizing the most engaging user experiences, and UX is a big part of it. Tony suggests, “I’m not sure we’ve found the sweet spot yet with tap, touch and swipe as we did a few years into the smartphone. We don’t understand the ergonomics completely for how to interact with this content, yet” — a rallying call for the industry to design experiences that can be more intuitive and sticky.
An exponential increase in high-speed information and high-quality content will turn locations into experience centers.
At the precipice of a 5G world is the promise of an exponential increase in access to high-speed information and high-quality content, with large-scale cities seeing an uptick of things that can happen because of the 5G effect. Marcie Jastrow, Senior Vice President, Immersive Media, Head of the Technicolor Experience Center, Technicolor, speaks to the connective nature of XR as a medium, potentially revolutionizing areas such as e-sports and concerts by unlocking access for billions of fans to shared live cultural moments, and also in education, where “learning through doing” can become increasingly democratized.
Jenny Rodenhouse, Director of Immersion Lab, ArtCenter, raises critical questions around points of access, with access to high-speed information only reaching some parts of the world, and not others. So too, “the collision of content on top of locations” will present a stream of possibilities and challenges alike — from how to signpost virtual content and let people know what is there to be discovered, to how to work with existing infrastructures across urban design.
Our changing relationship with technology can also deepen and enhance our relationship to place, providing more context into and emotional connection with the world around us. Leila Amirsadeghi, XR Pioneer and Co-Creator of The Unreal Garden, speaks to how new capabilities made possible through 5G — high-bandwidth rich streaming content and real-time, low-latency interactivity — “will enable locations to become experience centers.” Think of virtual travel experiences making it possible for anyone to visit the wonders of the world, the NY Times’s 360-degree investigative journalism, or the growing range of out-of-home experiences layered on top of the world around us, offering opportunities to engage more deeply with the world around us and cultivate a more socially connected society.
If the last era was about content, the next will be about context.
The convergence of XR and 5G will further enhance our ability to effectively convey ideas visually, spatially and quickly.
M. Pell, Author, Envisioneer and Design Lead, The Microsoft Garage, suggests that down the line, legacy media like images and video will be embedded into spatial interfaces that present information in more compelling and clear 3D formats.
Do you like to be read the news, but shown a recipe? Do you want to be able to access spreadsheets on-the-go on easy-to-read, virtual screens or watch your favorite sports game at your lunch break on a remote screen, zooming in and out as desired?
M. Pell tells us this will all be possible in the not-so-distant future, as auto-tools become increasingly capable of self-generating multiple formats on the fly, from automatically extracting a relevant executive summary from a longer document, to converting a static text description into a dynamic image or 3D visual. Further, he prognosticates that what kind of content gets served up in a specific moment will be tailored to your unique needs on that occasion, and how best you process information/content. As AI becomes increasingly powerful, the world of information will become increasingly personalized, making content available to you, how you want to consume it, in any particular context. And with the adoption of 5G and edge computing, we’ll have the necessary tools to process these media-rich experiences anywhere, on any device.
Asking us all to recall the first time we typed into Gmail and it knew what you were going to type, he suggests that just like we have spelling and grammar checkers now, we will have sentiment and emotional checkers. For example, Before you hit send on that Tweet, did you imagine that it could be sort of offensive, before sending? As M. Pell puts it, “Just on the horizon is pensiveness, sensitivity and appropriateness, and soon we’ll be dealing with an entity, not just a tool” — with XR as the “tip of the arrow” for bringing deeper emotional connections with technology to the mainstream.
We’re shifting away from a static world of information to a dynamic and highly personalized interaction with content.
A mindset shift towards human-centered design can enable a 5G era to overcome certain ills of the Internet era.
New interfaces open up new possibilities not only for more intuitive interactions, but also for more inclusive representation. Amy LaMeyer and Martina Welkhoff, Founding Partners at The WXR Fund reference issues in AI bias, such as how Voice Assistants struggle to recognize English as a Second Language accents, or how facial recognition performs best for white male speakers. As we move to thinking and creating in 3D rather than 2D, creators can aim beyond engagement rates and instead think more deeply about who they are creating for, and how to leverage the abundance of data we have to make experiences more personal, useful and enriching — asking how our innovations can offer value to end users, and what values they are re-enforcing.
Cortney Harding, Founder and CEO, Friends with Holograms, a leader in creating AR/VR/MR training experiences, shares one of the core principles behind designing with inclusivity in mind — designing for the least technical person in the room. “Since the technology is so new, the barrier to getting people comfortable using it is already high, so anything that makes it more intuitive and easy to use can only help.” Adaora Udoji, Director of Corporate Innovation & Entrepreneurship, RLab, builds on the importance of offering inclusive and accessible experiences that can impact our lives in meaningful ways. For Adaora, one key area for XR innovation is in healthcare, where virtual education can speed up how medical students understand basic concepts and get experience practicing complex procedures in a risk-free environment.
Adaora highlights how 5G can act as a great equalizer in certain ways, overcoming current barriers such as hardware costs that restrict who can create new software and products. Ori Inbar, Co-Founder/Managing Partner, Super Ventures and Co-Founder/Executive Producer, Augmented World Expo, encourages us to think about how it can level the playing field among tech titans too; while Google might currently be leading the charge because of its advanced processing capabilities, “the traffic we’re seeing today for processing XR is not comparable to what’s to come,” and 5G can set up multiple platforms for success across content and commerce experiences.
Thinking about the above-mentioned abundance of data, Ori also emphasizes the importance of not underestimating issues of privacy. “With spatial computing, when everything you do is captured on camera, concerns are going to expand a million-fold.” He challenges us to think of solutions; instead of selling our individual data, what if brands could aggregate data as collective knowledge? Multiple experts speak to the importance of routinely discussing how much data tech giants will have about our lives, and how to ensure more control over how we consume information, communicate with others and cultivate social connection.
As Amy LaMeyer suggests, “Let’s take what we’ve learned from the growth of the internet and related technologies and apply it to the expansion of spatial computing.”