AR & The Retail Sector: A Conversation
In the recent past, AR in the retail sector was mostly “try before you buy” experiences. You could, for example, put virtual furniture in a room, change fabrics, etc. Then came the pandemic. Brick-and-mortar outlets will, for now, operate with social distancing in mind. They’re in survival mode. What role can AR play in saving the retail sector? Or at least softening the economic shock? I touched on some of this in a video chat with Jennie So a few weeks back. The conversation below continues the initial chat.
Jennie helped scale physical retail for luxury brand Burberry’s markets in Australia, Southeast Asia, Japan and Travel Retail. Her background is in environments, computational thinking, design and enterprise. Always looking to connect the many dots between disciplines and cultures, she distills her observations on HCI, AR/VR and UX/UI, to rethink human experiences. Check out her articles here.
George has built on a design foundation begun at Sony, refined at Mozilla, and that continues with Project Gazzera. It’s a foundation backed by more than 20 years in UX design. His articles about immersive narratives reflect a broader interest in helping designers to understand XR fundamentals. And how to apply that understanding to Business, Education, and Entertainment.
Experience
George: Most retail AR experiences have been transactional. A few have been informational. 8th Wall, the XR technology, has a presentation, entitled “Augmented Reality Will Save Retail,” that’s available here. It shows in-home examples that approximate the in-store experience, traditional AR information display, and gamified experiences. It is a good snapshot of AR in the retail sector.
But the presentation is also an example of carryover, explained in detail here, which is bringing techniques and concepts from older forms or experiences into XR. The more carryover experiences there are in a business sector, the more AR must develop before it becomes natural, fully integrated into that sector. Will standalone transactional experiences go away? No. However, their relationship to the overall retail experience will change because the experience will be different from the pre-pandemic ‘old normal.’ What’s your perspective on this, Jennie?
Jennie: This is such a great point for exploration, George. Interesting observation there on the principles ‘carried-over’ from film (which I also studied way back when) I can see why this is an easy path to follow. Cognitively, we will always look for familiarity, engaging better with metaphors that we comprehend or have experienced before. In the early stages of any emerging technology, this is the least path of resistance for market acceptance.
I’m optimistic that as humans, with our openness and cognitive powers, we’re also able to handle way more than small increments in novelty. A few caveats : 1) there needs to be a conscious understanding of the limitations, strengths and weaknesses of XR, as mentioned in your article, and 2) using new tools that are appropriate for any situation. Just because we have something shiny and new doesn’t mean we need to use it, and if we do, we should do it pretty well.
George: Yes. you hit on two major points. Designers, and anyone who engages with them in experience creation, need to understand XR strengths and weaknesses. But also where differences, for example, between wearable and handheld AR, don’t matter in some use cases. Understanding starts with analyzing the critical interlocking areas: the spatial, the psychological, and the emotional. They are in a practical sense “the critical three” because they define the experience. And in retail, they define the brand. And the response to it. Understanding must reach a critical mass before ways of thinking that are natural to XR replace carryover thinking.
Your other major point raises a question. “Is a given experience something that’s novel, or a novelty?” The former will enjoy widespread adoption. And will move from the catch-all “experience” to something that has a descriptive name. The ‘novelty’ will remain a novelty. Or will become a historical curiosity.
Platform
George: For me, the term ‘phygital,’ a combined physical and digital brand experience, is equivalent to ‘motor carriage,’ circa 1900. It helps us conceptualize something that is not yet common. ‘Phygital’ will give way to something like ‘spatial presence,’ which will mean brick-and-mortar, some form of standalone AR, or AR within a brick-and-mortar space. The distinctions will only matter in how the consumer responds emotionally and psychologically. In short, Retail will shift from where we experience the brand to the brand as an experience. And will have a transactional component. Though transactions will be less the core focus than they are now. Your opinion, Jennie?
Jennie : Absolutely with you on that. The more we focus on designing the emotions and interactions vs focusing on the platform, the more innovative we will be. Noticed many of the initial examples on AR retail applications appear to be either functional or ‘cute’ — I’m optimistic we can do way more than that.
Exploring with respect to other disciplines and philosophies, such as film, allows us to stretch the imagination. There are many new ‘crafts’ that have emerged recently in computational design that are really worth looking into — poetic computing is one example.
George: Computational design has some overlap with AR. Both deal with human interactions in specific spatial contexts. And rely on computational simulations to do so. AR design professionals can in some cases apply computational design methodologies.
For retail, the result can be more meaningful experiences. But immersive branding requires understanding AR at the most basic level, which has implications for the consumer’s emotional, and psychological, engagement. This is where the medium of film can help us. We should look back to when creatives sought to systematically understand it as an art form. By that I mean to know it as thoroughly as classical composers understood music.
In the early 1920s, Russian director Sergei Eisenstein asked, “What is the smallest unit in a film that contains the essence of that film?” The answer was ‘the frame.’ It shows relationships between characters, location, etc.
In AR, what’s the equivalent to the frame, when what you see depends on where you look? And there’s no fixed composition that illustrates the relationship between, say, a digital character and the person having the AR experience? The answer matters because from the smallest AR unit we can build XR languages, akin to Film Language. And use them to guide us toward solutions for retail. By the way, AR does have analogies to the frame. And just as the frame is the basis for the Montage theory of film, the AR analogies are at the heart of XR theories.
Anyone involved with AR design should also examine Theater, from any tradition. Why? It deals with human interactions in 3D space. Simple gestures, the distance between two characters, have psychological meaning. Theater suggests enough reality for you to “suspend disbelief.” And enter a different world. This is done in part through set design. AR is no different. A wall transforms into the animated background of an AR experience. The wall is like a stage backdrop. And the store around you is a kind of performance space, where you (the customer) are the hero, enabled by the brand.
Jennie: Really appreciate your note about gestures, interactions, distances, and hence space. Whether in a store setting or an other-worldly construct, distributed cognition in this context is quite powerful as well, I think of AR as expanding the experiencing canvas exponentially. I’m aware this conversation is on AR, but can’t help but wonder what happens in VR — when the platform matures and allows our canvas to expand, ie we’re talking about a digital twin of our existence! In this case, film becomes a new medium within it? Or how else can we see the same theories apply or will they?
George: They will apply. But until headsets are indistinguishable from wrap-around sunglasses, I think VR in retail will find success mostly with luxury brands. One reason is headset cost relative to the price of what’s being experienced. The other reason is that we’ve yet to see everyday uses for VR that are neither entertainment nor work-related. But a luxury brand could present a VR experience as a near-exclusive, precisely because VR is not common in retail.
The future
George: AR-based transactions will exist within brand experiences that will be based on traditional concepts. But most transactions will be realized in ways that owe nothing to tradition. Brands will continuously engage the consumer instead of through the time-limited campaigns common today. This is critical for brick-and-mortar brands because continuous spatial presence, via AR, will mitigate the loss of outlets that may have been permanently shuttered. Or have restrictions on them due to social distancing. We’ll likely see crowded stores at some point. However, the pandemic will have left its imprint. One will be AR as so natural to many retail experiences that it’s no longer a novelty. What are your thoughts on this? And what might you say to anyone involved in store design?
Jennie: This is absolutely a time of convergence. First, we will benefit from zooming out from the ‘how-to’s and back into the ‘why’s. People are smart, and will know if a new tool is built for hype or to appeal on social media. The winners will be brands that know their customers deep down, demonstrate what they truly represent, and the rest will follow. In terms of the AR medium itself and how it would play out in the retail sector, I can see narratives becoming more sophisticated — by tapping into the user’s individual styles and thinking patterns, not a one-size-fits-all, creating unfamiliar, unique and blended environments with gestural and voice interactions. When warranted, this is definitely one way to push and extend the traditional canvas in terms of an either-or scenario (physical space or digital devices) What we know of as ‘store design’ will morph into ‘interaction design’, a broader field composed of many different disciplines.
George: Exactly. Interaction design may become the overarching discipline because in a world where the brand is an experience, ‘experience design’ as currently understood creates confusion. This shift in terminology will represent the broader shift not in how we define retail. But in how we express it. ‘Retail’ will have a dictionary definition. It’s not what we’ll experience in the real world. Experience types will fall into patterns, as with any interaction design. However, we’ll see wild variations driven by the absolute need for brands to differentiate themselves. The design challenge will be to ensure that critical interactions within experiences are unique, yet practical.
