Grocery Shopping in Virtual Reality
An Exploratory Exercise into the Feasibility of VR Grocery Shopping
Written by Megan Duyongco, Johnson Ma, & Kyle Tizio — September 5, 2021
Background
We embarked this quarter on an exploration of the feasibility of grocery shopping within the world of Virtual Reality (VR). More specifically, this would be an exercise aiming to understand how users would interact within a digital grocery shopping medium based on their experiences in the real world.
The emerging technology of VR is new enough that it still poses challenges for users. It is intuitive yet clunky; it leverages our pre-existing mental models of how to interact, yet it utilizes new virtual tools and capabilities and occurs within a virtual space. What we are capable of in VR environments is often an emulation of real human behavior, yet the concepts are new to us, as is the case when learning any new system.
Consider this project our lens for exploring how users might try to grocery shop and interact with the capabilities of VR — is it a viable option for companies to try and implement or is it better off as a standalone platform?
The Design Challenge
Government quarantines and social distancing mandates, enacted as a result of the COVID-19 pandemic, have accelerated our dependence on digital interactions. Consumers have become accustomed to tech as a replacement for once vital in-person interactions. From the way we work, shop, or enjoy entertainment, consumers have developed an expectation that nearly everything can be done online.
While it would be wise for retailers to study the present situation, the leaders of the industry must also look to the future. VR is no longer the future tech only found in science fiction, rather it has become a tool readily available to the average consumer.
Goals:
- Determine the viability of a 3D platform for grocery shopping
- Personalize the shopping experience unique to the individual
Who We Are:
We are Megan Duyongco, Johnson Ma, and Kyle Tizio — three User Experience Design students completing a UXD Certificate through UCLA Extension (UCLAx). This summer, we had the opportunity to complete a Mentorship under Adam Fischbach, our instructor from UXIII, a class throughout which we utilized a Service Design approach to recontextualize the car-buying experience.
When Adam mentioned the possibility of taking on a mentorship group for eleven weeks, we jumped at the opportunity. We were given a loose set of constraints for the work to come and told that the project would have to fulfill two criteria:
- It must be Virtual Reality-based
- It must incorporate gestures and touch on human interaction in this emerging technology
The rest would be up to us. Our group was new to creating VR environments and immersive experiences, so we quickly got to work orienting ourselves in research, aiming to determine where the technology was being used, the room it possessed for growth, and the viability of bringing it to consumers for implementation in their lives.
Discovery
From the onset of the project, our goal was clear: create and determine the viability of a VR grocery shopping experience. We began by putting together an assumption grid to plot our preconceptions of what grocery shopping in a VR environment might entail. By charting a visual representation of these assumptions, our team quickly identified those that posed the most risk and consequently the areas to test first.
In the chart above, we individually listed our assumptions, and then as a group, we placed each of them onto the right-hand grid. We grouped similar items and labeled the category that would describe them, allowing us to identify our certain high-risk assumption areas and our uncertain high-risk assumption areas. Having narrowed our scope, our next step was to conduct research to challenge these assumptions.
Research
Searching for Direction
We chose user interviews, specifically moderated user interviews with open-ended questions, to probe responses from customers for deeper insight. By asking open-ended questions, we had the benefit of allowing the user to guide us towards issues that we might have overlooked in the assumptions grid.
Qualitative Research
Before we could conduct user interviews, we needed to answer two questions:
- Who are our target users?
- How do we recruit participants?
We surmised our initial target audience was users whom our product directly caters to, namely VR headset users who also grocery shop. To expand our representative sample, we brainstormed and identified potential second-degree users as those who tended to be busier than time rich. This secondary group included, but was not limited to, professionals, parents, early adopters, Covid conscious, gamers, and grocery workers.
Once we identified our target users, we needed a method of recruitment. We weighed the cost-benefit of outsourcing participants to a usability testing firm but ultimately chose to recruit via friends/family and cold calling. At this early stage in exploratory qualitative research, it was faster, cheaper, and likely yielded similar results.
In total, we interviewed eight users, each of whom was screened to be suitable for our study. From the data gathered, we analyzed each response, grouped them with similar responses, and coded the groups under a single banner. For example, the pain point of unreflective inventory quantities between online and in-store shopping was summarily categorized as inventory.
We were able to validate four of our seven assumption areas: quality, immediacy, inventory, and human interaction. Quality was confirmed as high risk, given that nearly all eight users had poor experiences ordering fresh fruit/produce online. Interestingly, users considered a single poor experience as justification to stop ordering fresh produce/fruit entirely when done through a delivery service.
As for immediacy and human interaction, both were considered low-risk assumption areas, given that six of our eight users indicated these as non-issues and satisfactory as is. The immediacy of purchasing in-store depended on distance, and so long as the distance was convenient, users were willing to travel. Further, users indicated that online orders had reliable time windows across all platforms (e.g. WholeFoods, AmazonFresh). Additionally, online orders had the option for immediate delivery at the cost of an additional fee. For these reasons, users confirmed our assumption that immediacy was a low-risk consideration in grocery shopping.
Human interaction was also a low-risk factor since users indicated that they typically kept to themselves when grocery shopping, save for the occasional help from store employees. A surprising detail was that while users kept to themselves, this was not necessarily indicative of them wanting to be by themselves. Nearly all users reported preferring to shop with a company, enjoying the indirect presence of others. One user, identified as “RM” for the sake of privacy, listed human interaction as the main draw for returning to in-store shopping once Covid subsides.
While our assumptions on quality, immediacy and human interaction were validated through data collection, the same could not be said for inventory. We considered inventory to be low-risk, yet our users (five of eight) indicated inventory disparity as a major pain point for both the in-store and online shopping experience. For in-store shoppers, this meant being unable to find an item and having to go to a second (or even third) store, while for online shoppers, inventory disparity meant online products didn’t reflect the actual quantity in the store, and thus alternatives were needed. In one example, user “RM” placed a delivery for hamburger items to be used at a barbeque, yet once he received the delivery, he noticed that the buns had been left out. In reviewing the order, “RM” realized that he had failed to provide an alternative if the requested buns weren’t in stock.
Having analyzed our qualitative research data, we created a persona to embody a cross-section of our users who preferred to shop via the traditional in-store method.
Quantitative Research
With three more assumption areas (beginner difficulty, sensory, and pricing) to challenge, we needed to expand our sample size. Thus far, our users included those with the direct grocery shopping experience, either in-store or online and fell within our demographic range. However, we failed to sample users with VR headset experience. This needed to change, and so we turned to quantitative research methods to gain insight into the cross-section of VR headset users and those who shop for their own groceries.
Since we wanted to sample a wide spectrum of VR headset users, we created a survey to gauge interest in a VR grocery shopping experience and administered it to various VR online communities on Reddit. The communities that participated included r/HTC_Vive, r/Oculus, and r/VirtualReality.
We received 35 responses, which were then compiled into an excel sheet to be used to challenge our remaining three assumption areas: pricing, sensory, and beginner difficulty.
With our data collection complete, we began to validate our assumptions. We had identified pricing as low-risk, yet the responses from our users suggested otherwise. A majority of users (15 of 35) reported affordability as the first most important factor in grocery shopping, followed by quality.
Furthermore, when we cross-referenced our survey data with our user interviews, four of eight participants also cited price discrepancies as a pain point. For these reasons, we were incorrect in our assumption that price was a low-risk factor in understanding grocery shopping motivations.
We then considered the idea of the sensory experience, which we had marked as a certain, low-risk assumption area and defined as having the ability to be touched, felt, and understood with the senses. We believed sensory would be low-risk under the supposition that VR would offer an immersive experience for the user and therefore add — not take away — from the user’s experience. Our survey data confirmed our assumptions. When we polled the question, “Why or why wouldn’t you be interested in VR grocery shopping?” 6 of 13 respondents to the “why” indicated interaction and the ability to inspect items at every angle as the main draw to the technology in this context.
We categorized the area of beginner difficulty as a certain, high-risk assumption area, believing that users unfamiliar with VR might be easily overwhelmed and therefore discouraged from using it further. An analysis of our data found a positive correlation between experience level in headsets to interest in VR grocery shopping. While correlation is not causation, we had enough evidence to validate our suspicions that the perception of a steep learning curve, whether truly difficult or a preconception, would have a deterrence effect. The caveat would be VR experience. With this knowledge in mind, we would later ideate on bridging the gap in our next phase.
Once we completed and analyzed our quantitative research, we created two personas to align on user empathy and drive our design decisions moving forward.
In summation, four of our seven assumption areas were high risk: quality, inventory, pricing, and beginner difficulty. These four areas would be our focus points around which to ideate, given their impact on the user experience. The remaining three assumptions (i.e. immediacy, human interaction, sensory) would have elements pulled into VR since we wanted to retain what users already prefer.
PROBLEM STATEMENT
As businesses adjust to new online consumer behavior and preferences, grocery stores have struggled to offer an enjoyable shopping experience from the comfort of one’s home.
A VR grocery shopping experience should offer not only the benefits of online shopping but also the satisfaction the customer comes to expect when purchasing in-store.
Ideation
Seeds of Ideation and How Might We…?
As we began wrapping up our initial rounds of research and moving into ideation, we thought it would be important to anchor ourselves within the framework of recontextualizing, rather than reinventing, the grocery shopping experience. How could we do this without completely overhauling peoples’ experiences buying groceries, and more importantly, how could we leverage the mental models that people already have concerning grocery shopping to offer them a new way to complete a task they’re already accustomed to doing?
With these grounding principles in mind and working within the constraints of our eleven-week timeframe, measured VR capabilities, and the exploratory framework of this project, we leveraged our insight from user interviews and targeted user surveys to develop a collection of guiding “How Might We…?” prompts.
In creating these, we pulled from common user pain points of the in-store shopping experience as well as delivery service experiences. Among the responses, we found that users overwhelmingly noted the following as ranking high on the list of what could be done better in grocery stores:
- Wayfinding
- Efficiency and time productivity
- Item availability
It is also important to note that many users reported the lack of quality and freshness in the case of produce, although this was noticeably more prevalent in users who reported using delivery services more frequently, indicating a breakdown somewhere between the process of confirming an online order, that order is fulfilled at the store, and then be delivered by a third-party service.
Variation Through Sketching
Our main drivers of ideation were sketching exercises, through which we aimed to generate a wide range of ideas that would touch on some of the more central “How Might We…?” prompts that we’d developed. We conducted multiple rounds of Crazy 8s as well as our own take on the 10 plus 10 methods, which looked more like 9 plus 9 for our group of three. As per our experience in prior classes, Crazy 8s encouraged us to go wide in areas of concern; 9 plus 9 allowed us to go deep and drill further into ideas in these same areas.
The most important result of our early sketching was alignment on the vision of a reimagined grocery shopping experience. Our work encouraged us to distance ourselves from the idea of simply taking a grocery store and placing it into an immersive VR environment. How would the user move, if at all? What might their cart look like? How could we incorporate natural gestures to facilitate the process of choosing, confirming, or manipulating items to be bought? Could we push personalization and tailor this experience for each user?
Iteration
We arrived at what we would begin to call the Grocery Carousel, a Lazy Susan-like array of columns rotating around a fixed, central point. These columns would, in essence, aim to mimic the aisles of a grocery store — as if they were vertical instead of horizontal. Our first iterations imagined these as grid-like displays that could be rotated, from which a user could grab items, inspect them, and add to their cart just as they would in a real-life grocery store.
This idea quickly demonstrated its ability to become overwhelming, and we realized the need for successive “levels” of this experience — namely, allowing a user to progress through varying tiers of detail when viewing products. Trying to draw as close parallels as we could to a traditional, in-store experience, we imagined the following, with each step bringing the user to a more detailed viewing and interactive ability:
- General Carousel View — All item categories (columns) represented by a selection of groceries (not all) from that category, likened to one’s view when first entering a store and seeing multiple aisles in front of oneself
- Detailed Shelf View — As if the user had walked into a particular aisle or section at the store and now had the ability to view all of the items within it
- Product Detail Page — A detail page of item-specific information which would appear if a particular item was selected for further inspection
- Sandbox — More of a “mode” than a page, the ability to manipulate immersive 3D assets and retain the ability to add these assets to one’s cart (ie. a watermelon halved or quartered, of which any or all parts may be purchased)
With a basic understanding of how this grocery apparatus would operate, we convened to discuss what the architecture of this VR platform might look like. We generated a sitemap to better understand how deep the experience would go and the latitude that a user would have in moving about this immersive environment. On its face, this highlighted the difficulties of mapping out a VR experience in a 2D method; this realization led seamlessly to our next endeavor, which meant grappling with the ability of VR to teleport its users around an experience and the sense of presence, which Mina C. Johnson-Glenberg referred to as the “first profound affordance” of Virtual Reality in her 2018 study Immersive VR and Education: Embodied Design Principles That Include Gesture and Hand Controls.
Navigation & Teleportation
We understood that we weren’t simply taking a grocery store and placing it in the context of a VR headset — after all, we felt that having a user have to manually teleport through the many various aisles of a store to find a selection of grocery items would be too much. As such, we envisioned a scenario where the content would be, in a sense, delivered to the user and not the other way around. How could we reduce the number of transition periods between levels, and how could we incorporate teleportation to avoid forcing users to work through the experience in a very traditional, linear, step-by-step manner that one usually uses to work through a series of menus? After all, we are talking about a 3D, fully immersive space.
To demonstrate this idea, consider one’s place in the ecosystem when on a product detail page or the detailed shelf view of the dairy section — how might they be able to arrive at a view of the frozen foods without having to manually back out of where they are, spin the entire grocery carousel to the desired column, and then dive once again through successive levels? Enter the idea of a Smart Assistant, our alternative to the omni- yet optionally-present Siri or Alexa — willing and able to teleport one to the precise area desired. Additionally, adding a tab of main categories (those represented by the columns of the carousel) would allow users to reduce the steps needed to move around this experience level-by-level and lessen the likelihood of simulator sickness, which has been known to occur in these types of environments, especially when users are subject to excess motion.
Prototyping
Prototype Fidelity
As this was our first exploration into recontextualizing the grocery shopping experience in VR, we had to evaluate the time it would take to create a VR experience, the constraints of free VR prototyping software, and remember that our main priority was research. We decided on a low-fidelity prototype for exploratory purposes which allowed us to focus on content, rather than visual design and gain a more holistic view of the user journey since our objective is usability.
Leveraging DraftXR & AdobeXD
We managed to find a great plugin for AdobeXD, which allows users to leverage 2D assets yet publish drafts within a 3D environment. Though it lacked the true functionality and feeling of presence within a VR environment, DraftXR gave us the mobility to mockup most of what we had envisioned for this project and the majority of our “Happy Path” user flow. We were able to prototype within XD, allowing users to click-through screens and progress through the grocery shopping experience. Together with screenshots of our work in ShapesXR and prototyping functionality, we would be able to ask users to allow for a bit of imagination when progressing through the flow — and though we lacked the ability to actually utilize gesture control via the VR headsets, a series of click-through screenshots demonstrating hand control and gestures worked well enough to demonstrate the functionality we were imagining.
Prototyping Gestures in ShapesXR
Currently, VR leverages the use of dual controllers, one in each hand, for point and click interactions within the headset. While controllers are precise and easy to learn (since they utilize the basic principles of a computer mouse), this method nonetheless feels like a relic of the past. Facebook, the parent company to Oculus, shared our sentiment and released gesture controls in 2020 for the Oculus Quest. As it relates to our project, we wanted to leverage the new technology and decided to ditch the controllers, designing for a truly new-age experience, one that was controlled by only the body.
Since our VR headsets were the Oculus Quest 2, it made sense to import native gestures that Oculus users were already accustomed to, specifically, point and pinch to select. With basic selection established, we turned to the VR gaming industry to explore more engaging ways of interacting in headsets. We chose the gaming industry because video games are designed to be immersive, entertaining experiences. Video games must hook and hold onto the gamer: this means pushing the limits of either storytelling, graphics, mechanics, or a combination of all three.
We were heavily inspired by a game called SUPERHOT, wherein a player must destroy oncoming waves of enemies by punching, throwing objects, or shooting weapons.
For a SUPERHOT player, there is no pointing and clicking; instead, they must reach out their arms and grab objects as though they were truly in front of them. We were intrigued by the gameplay mechanics of direct object interaction and felt that it translated seamlessly into our VR grocery shopping experience.
While faraway objects would be selected via the point and pinch gesture, we emulated the direct interaction experience by repositioning the user’s line of sight to objects at an arm’s length. By closing the distance between user and object, the former would be capable of clearing the distance to reach forward and directly grab ahold of objects. In the screenshot above, the user can be seen grabbing a yellow bag of candy; just as easy as it is to grab an object directly, so, too, can she select objects further away. To select the can of chips located on the top shelf, the user would raise her hand to the object and pinch her fingers to snap the item into her palm.
To reinforce our use of hand gestures, we based our design decision through an article published in Applied Sciences by Guerra et al. (2019), Hand Gestures in Virtual and Augmented 3D Environments for Down Syndrome Users, which sought to test the most commonly used interactive 3D gestures between both Down’s Syndrome (DS) and neurotypical participants. We chose this study for two reasons: First, uniform VR usability guidelines are not fully established and can therefore increase cognitive load, even among experienced users. Second, accessible designs can induce what is known as the curb-cut effect: although initially designed to benefit the disabled, these ideas produce a ripple effect of benefit for a much larger group.
The Guerra study suggests that there is no statistical significance in successful execution and time for the grab-and-drop gesture between DS and neurotypical participants. Furthermore, Guerra noted that due to limitations in motor skills, 3D gestures should involve the whole hand as opposed to fingers. The impact of the insight from Guerra's study on our work was twofold: first, it validated our design decision to incorporate direct interaction, since it established that cognitive ability would not be a critical performance factor. Second, to expand 3D accessibility to all users, we would iterate upon the native Oculus gesture of point-and-pinch to select, from thumb + index finger to thumb + all four fingers.
With Guerra’s findings in mind, we expanded the accessibility principles into our interactive Sandbox Mode. While designing the interface wherein users would interact with the product via cutting, turning, throwing, etc., we noticed two immediate issues. First, the user’s hand would shake after the prolonged raising of their arm; second, cutting objects required precise measurements, which, without the use of guidance, resulted in imperfect or accidental actions.
To alleviate the two issues listed, we once again turned to the video game industry. In an online game of billiards, for example, a dotted line emerges to guide players as to where the ball will travel based on the cue’s positioning. The dotted line is easily understood by players of all levels and conveys enough information for an accurate strike.
By leveraging the dotted line functionality and the use of the hand (not finger gestures), we believe it would resolve the shaky hand and imprecise tracking issues. The dotted line creates a more accurate, true-to-intent guideline, thereby reducing the attempts needed for the desired outcome. Furthermore, by utilizing a large sweeping hand motion, users would reduce the time spent with their arm raised as opposed to smaller finger gestures.
Prototype Design
As this experience would rely solely on hand tracking for interactions, we needed to figure out the visual or audio cues our test environment could afford, sans the tactile feedback that one gets from the controllers. We had spent time debating the potential benefits of integrating visual cues that leverage users’ working knowledge of touch gestures on hand-held devices, such as that of an iPhone app beginning to shake when tapped and held, signaling that it can now be moved or deleted. Another major consideration was the seamlessness of the experience — for example, how could users unlock context-specific gestures within a single scene without having to rely on a settings menu to enter a different mode? Unlike hand-held devices, the significant advantage VR environments offer are representations of physical space. We weighed the pros and cons of an idea to define a specific area where objects could be “anchored” to unlock a set of gestures available for that particular item; we would call this the “Sandbox,” and it would allow further manipulation of items based on the proximity to the user.
Ultimately, we integrated the following, believing that these additions would allow users to take the most advantage of the environment and minimize interaction costs:
Highlighted objects: a visual indication to users that the system has recognized that grocery products, shelves, and the shopping cart are interactive and have been selected.
Voice command: to be used with the smart assistant who performs a role similar to that of a virtual assistant like Siri or Alexa, and emulates the presence of a grocery store employee.
Buttons and UI: considering the number of products that are available in grocery stores at any given time, we relied on buttons and panels to indicate that by selecting them, more information or 3D objects that were not currently within view would become available.
Creating the VR environment and life-sized 3D objects in ShapesXR pushed us to think about the scale of the items and how many of them to display at once. In addition to designing components and objects to be within arms reach for direct interaction, or close enough to be brought into arms reach with minimal effort, we aimed to keep relevant objects in the line of sight, affording users a reduction in unnecessary head movement that could result in fatigue or motion sickness.
We set out to create prototypes according to the key stages we identified in our user flow.
Carousel — Home Shelf and Category Menu
Key Features:
- Get an overview of all the categories and browse within them
- View recommended and commonly purchased items with Home Shelf
- Poke to select a button that takes users to a detailed shelf view showcasing all items in the category
- Swipe to rotate individual shelves to view items on each
- Grab item to enter Sandbox mode
- Grab item to add to cart
The carousel view would act as the opening scene for users. It would act as a physical main menu from which one can browse, as though browsing through grocery aisles in a store and allow for direct interaction with objects.
Sandbox Mode
Key Features:
- Slice to adjust the quantity of an item
- Grab item to add to cart
This would be our main area of opportunity to explore new gestures and interactions. A study published in Applied Sciences, Design of 3d Microgestures for Commands in Virtual Reality or Augmented Reality, by Li et. al (2021) found that participants are willing to learn new, unfamiliar gestures for some commands instead of more familiar ones, something we wanted to test with users.
Broccoli Rob — Smart Assistant
- Voice-activated smart assistant available throughout the shopping experience
- Voice command for locomotion
To make the experience accessible and available to someone in a stationary position and minimize any discomfort one might feel by needing to navigate around the test layout, we envisioned that users could employ voice commands for teleportation to various stages in the shopping experience.
Detailed Shelf View and Checkout
- Prototype products and portions of the user journey in DraftXR to create photo-realistic representations of our imagined 3D object fidelity
We utilized the DraftXR plugin to prototype portions of the user journey and create photorealistic representations of our imagined 3D object fidelity. For prototyping, we created a 4x9 array to represent a store shelf and the items on it. A user’s cart would always be right in front of them and thus afford product additions simply by grabbing items from the shelf and releasing them into it, much like an in-person experience.
User Testing
Goal
At last, it was time to put our work to the fire with real participants. We were interested in exploring how users would perceive this recontextualized take on a common activity. We set out to validate the decisions we’d made among a participant pool similar to those with whom we’d aggregated our research data, namely peers and those with prior headset experience.
Usability tasks were created with these specific goals in mind:
- Understand if the environment afforded users adequate visual cues for how to manipulate and interact with the 3D objects and interfaces
- Test user ability to navigate around and understand the layout
- Explore how users would utilize context-specific gestures
- Unpack whether users preferred different gestures
Method
We created a tutorial video to familiarize users with the reimagined shopping environment, pointing out key features and gestures. This functioned similarly to how First Steps orients new Oculus users with their landing experience, or more commonly with product onboarding for 2D non-immersive interfaces. There were two main considerations for this:
- Most of the users we interviewed or surveyed did not have any extensive VR experience. The intent of the tutorial would be to prime them for what they could expect to see and encounter in the test environment.
- We wanted users to receive the same set of instructions and accompanying visuals. This would provide them as uniform a baseline as possible to account for any advantage with past VR experiences. It would also reduce the number of variables that could lead to confusion about how to interact in the environment. We were also interested in the extent to which they might feel the need to recall information if the testing environment did not provide easily recognizable prompts.
We conducted a hybrid method for testing our prototypes, guiding our participants through a user flow of the end-to-end shopping experience compiled in DraftXR. Participants were encouraged to think aloud and gesture as though interacting with the 3D objects directly.
After the test, we had users fill out a System Usability Scale to get each of their subjective assessments of usability. The average score was 72, placing the experience in just about average for usability.
Results & Feedback
In total, we tested our prototype on five users through a mixed-method VR format of 3D (in headset) and 2D (on a computer screen). We moderated the usability testing and asked our users to perform a series of nine activities, documented in our usability test guide. Our testing covered the user’s journey from login to checkout, broken into three phases:
- Homeshelf view to explore carousels
- Detailed shelf view to explore large-scale UI
- Sandbox-mode to explore gestures and natural intuition
The first two activities introduced the carousel homepage to the user and tasked them with placing a bag of candy into their cart. They would then navigate to view all snacks. Here, all five testers completed the first activity without issue; additionally, their body language was indicative of their success. All five testers gestured their arms forward as though they were grabbing the object and then placing it in the cart. We discovered that user navigation to view all snacks was not as intuitive, with only three of five testers able to complete the activity. Users cited the uncertainty of how to initiate navigation as their main issue. The two users who struggled were uncertain as to whether to click on the carousel titles, swipe with their hands, or pinch the chevron buttons.
Tasks three and four had users check the nutrition facts on a bag of chips and then place the item into the cart. This stage of testing explored the detailed shelf view and large-scale UI. While three of five participants were able to check the nutrition for the chips, all users struggled with the UI of a 2D interface on a 3D experience. Users especially struggled with text since they were unsure whether the text was written to be read or able to be selected. Furthermore, two users failed to place the chips into their cart, when moments prior they succeeded in placing a bag of candy. The users who struggled, voiced their concerns over the object’s distance, lack of response when motioned, or inability to discern if the cart was interactive.
Activities five and six tested voice control and the sandbox mode for gestures. Users were asked to activate their digital helper, “Broccoli Rob,” to initiate voice control for guidance. Once in sandbox mode, users were asked to cut a watermelon in half and then one of those halves to their cart. We anticipated users would struggle with this sequence of tasks, yet to our surprise, all users completed both activities and required very little assistance. It is notable that once users were familiarized with voice control, they routinely used or asked to use the feature for the remainder of the activities. Additionally, all users performed the full hand swipe motion with ease and little instruction. Testers appeared to act instinctively when cutting, with two users likening the activity to the popular mobile game Fruit Ninja. One of the users mentioned aloud, “I’d probably use my hand to slice it in half like Fruit Ninja.”
Activities seven and eight tested the Review Cart and Checkout stages. Users were asked to inspect their cart and remove the bag of chips before proceeding to checkout. Here, all users successfully removed the chips, but perhaps more importantly, they all motioned the same, if not an identical gesture of grabbing then throwing to remove the item. As for the checkout process, all users were also able to complete this activity, however, three of five users voiced concerns over security since their payment method had their information pre-filled. Further concerns on the checkout process include, or rather lack, a final order confirmation page for review.
Final Thoughts
Overall, this project was a lesson in challenging our design decisions from prior steps and considering how to adjust moving forward. The testing phase proved no different — due to imperfect testing methods and the inability to truly test the gestures we’d imagined within the VR environment, we found the space between success and failure on certain usability tasks was sometimes a bit too ambiguous, lacking the sort of insight that could propel us to revisit some of our decisions. What we did find valuable, however, was much of the rich insight gleaned as a result of our think-aloud testing. We’ll highlight some of the areas in which users struggled and noted them as points to be improved.
Broccoli Rob — We learned that our choice of Smart Assistant, cheekily named Broccoli Rob, might impose a type of pressure on users to make what is generally considered “smart” or healthy choices. Users were generally split on whether or not they wanted to know that the Assistant could be utilized at any point in the process, and thus have a constant presence somewhere in their field of view. In a couple of instances, users were under the impression that Broccoli Rob would check on them if they were not completing any actions. Some suggested that a feature that checks in on shoppers frequently throughout their experience could be helpful. This idea in practice could contribute to alert fatigue, but the value of incorporating an Assistant tool to simplify steps in the process has a large potential upside — this could include a more active role in moving between categories of groceries, checking out, adding items to one’s cart, or comparing between products.
Which Information to Display — VR environments are a highly immersive and stimulating experience, and so it’s a fine line between providing adequate levels of information for a user to progress and including too much. Some users noted the desire to be able to view prices from the Carousel View and Detailed Shelf View. We’re not sure this is the answer, as we iterated earlier on price inclusion and realized very quickly that it was a lot to view at once. It would be a nice consideration to view price when an item is picked up, or in the case of the cart, when products enter the field of view. We discovered a similar feeling among our participants when it came to Sandbox Mode — they were more confused than aided by the UI elements, so we removed them to return the focus purely back on the gestures.
Conclusion
Acknowledging Our Constraints
It’s important to note that this project was by no means exhaustive — in fact, in light of its speculative nature and our limited resources in prototyping and testing in VR environments, it was never going to be. Our inability to test gestures among our subjects in the headset, paired with being forced to rely on multiple mediums to articulate our vision for what grocery shopping in VR might look like, posed challenges for us. However, we feel that this is a great jumping-off point, and of course, just the start of where further research could lead in the space. Looking back at our work, it’s safe to say that we attempted to create a lens through which to view and understand the capabilities of VR in completing a task that most everyone does.
The Feasibility of Virtual Reality Grocery Shopping for the Masses
Put simply and as it stands today, a project like this is just not yet feasible for companies to develop while hoping for a substantial return. The interest for a VR-based grocery shopping platform exists but based on our research and testing, only in niche target segments, namely those already accustomed to spending time in headsets and those who utilize the technology in other parts of their lives. More broadly, this trend fails to extend to general audiences, as the barriers to entry remain too high and cases for day-to-day use too few. The headsets are still too clunky, expensive, and seen as a novelty when it comes to the general population, especially given the access to well-functioning delivery platforms like Postmates, Doordash, etc. which are native to the now-ubiquitous smartphone.
That said, as the technology develops and begins to shift from being seen as a novelty to a capable platform that affords a unique virtual immersion, we see the beginnings of where a project like this could go — take, for example, AmazonGo, a true grab-and-go alternative to traditional check-out processes driven by contactless convenience and “just walk out” technology. We don’t believe it is too far a leap to imagine a model that functions similarly, only purchased from the comfort of one’s headset.
Continued Exploration in the Space
An experience must be usable before it is enjoyable. To that end, we feel that we achieved the first part of this: recontextualizing the grocery shopping experience and prototyping a usable platform in a VR environment.
In the future, we imagine a platform capable of complete personalization. Consider the value of being able to include dietary preferences and restrictions, like setting alerts for foods with processed sugars or those that are high in sodium. We’d also like to note the exploratory benefits of interactive VR assets, such as understanding the size and weight differences between pieces of protein or the difference between a crumbly queso fresco and a more shreddable block of mozzarella. There is so much to be gained from the immersive level of detail that VR can provide, and we believe that the ability to interact, understand, and manipulate products for one’s order at this level will restore many of the intangibles that are lost in the transition to online grocery shopping.
