From 2D to 3D: What We Learned Designing an Augmented Reality App

Our experience using Torch for rapid AR prototyping

Julia Park, Interaction Designer | Chrisanthy Rebecca Surya, UX Researcher

When Pokémon Go launched in 2016, it quickly became the most popular augmented reality (AR) game. In recent years, AR technology has accelerated not just in the game and entertainment industries, but also in medicine, training, education, and other fields. And yet, we’re still in the beginning stages of this technology. The potential for AR is considerable.

At Favorite Medium, we’re always curious about new technology and creative opportunities, so our designers embarked on an AR project. We wondered: How would our 2D design skills translate to a 3D app? Does AR require a different kind of design process? What follows is an account of our experience and what we learned with rapid ideation and prototyping in AR.

For this project, we picked an activity that many people encounter on a daily basis: shopping.

The Process

Discovering Insights

We started with some desk research to understand the latest retail industry challenges and trends, and to ascertain how AR is currently being used in the retail sector.

Next, we conducted in-depth one-on-one interviews with Singaporean millennials (22–37) to learn more about their current in-store shopping experiences, the challenges surrounding it, and how it can be improved.

One-on-one interview with one of the Singaporean millennials

From these processes, we discovered three ways we can improve shoppers’ experiences based on their needs:

1. Help shoppers maximize their use of time in stores.

a. Inspiration is key. Shoppers don’t consider browsing to be wasted time if they can learn about new trends and products, increase their product knowledge, and feel inspired. Even if they don’t end up buying anything, a visit that helps them gain information and inspiration is worth it. However, brick-and-mortar shops often don’t accommodate this learning need.

b. Make it easy for shoppers to find their style. When they’re in a store, shoppers want to locate items that suit their personal style easily and quickly check the quality and fit.

• Shoppers prefer buying items that are similar to ones they already own because they feel comfortable with that style and already know that they’ll look good in it. Buying these items means they can mix and match with their existing items, and use them multiple times. This also means they feel that their money has been used wisely.
• Shoppers want to use their time efficiently. “If I can see new variations of the item I last bought quickly, that would cut down my shopping time.” — Participant #1, F, 28

2. Build emotional connections with shoppers.

a. A relaxing and restful environment. Shoppers often return to shops that feels “homey,” “welcoming,” and “comfy.” These types of spaces make them feel that shopping is not a burden and that no one is pressuring them to buy things, eliciting positive emotions. This emotional gratification attracts them to return to those shops.

• As Nathan Watts, the creative director of FITCH, told design:retail magazine in the Jan/Feb 2019 issue, “Emotions is a key component of effective physical storytelling…Engaging spaces often augment layers of sound, smell, and even taste. When combined effectively…they can turn a series of experiences into a compelling story.”

3. Help shoppers maximize their money — the best item at the best price.

a. Shoppers want to get the best item with the limited money they have. However, comparing items manually from shop-to-shop is time-consuming.

b. Shoppers seek expert reviews before shopping to check product quality and whether it’s worth their money. This is especially important for expensive items or items they can’t try or test immediately.

• “I’m not a compulsive shopper, I like to make informed, educated purchases.” — Participant #2, M, 29

Putting Things Into Context

After discovering these insights, we mapped out the user journey to have a clearer sense of the user’s emotional journey, pain points, and areas of opportunities.

We also came up with a set of How Might We questions, which were useful to ideate different concepts and ideas for our prototype.

After mapping these, we asked: Which of these pain points and How Might We questions can we address through AR? Which pain points are most severe, and which questions are more interesting to explore through AR?

We decided to focus on helping shoppers find garments that appeal to their personal style and quickly locate them in store, providing trusted reviews on products, and making the checkout process faster and smoother. We could not address all the insights since AR was not necessarily the best solution for some of them. Due to time constraints and limitations in the Torch app and/or AR technology itself, some concerns and questions didn’t seem well-served by AR. For example, creating a homey, welcoming, and relaxing shop would be better addressed through changes in the physical store themselves or even through virtual reality.

In addition to discerning which pain points were most severe, we also picked corresponding questions through which we could better showcase the potential of AR. From these processes, we narrowed down our problem statement for our project, which became the overarching question we tried to address through our concept:

How might we use AR to help shoppers get the best item for them easily and quickly in a shop?

Finding the Suitable Design Tools

We wanted a tool that would be easy and quick for us designers to use and that would allow us to create a prototype without coding. Based on our research we learned that we can create quick AR prototypes in Torch app with no coding. Torch also allows us to upload 2D and 3D assets, and integrate with Sketch or Figma, which was appealing considering that we are familiar with both. From viewing the tutorials on the app, we thought it would be quite straightforward to create interactions within the assets in the 3D space. So we decided to try Torch as our prototyping tool, and use Sketch to design our UI assets.

Crafting Our Ideas

Before designing the UI and prototyping, we created a storyboard to contextualize our concept and to further understand how users can use the product in-store.

Then we created low-fi prototypes by drawing the screens on a series of transparent sheets and placing them in sequence on top of an iPad. This allowed us to do some basic usability testing and validate our user flow in the actual, in-store environment.

From testing these low-fi prototypes in-store we discovered that:

• Without their own background layer, all texts are hard to read and see as they conflict with the store images captured by the camera.
• Map thumbnail of location of item in-store might help users get a sense of where the item is in store.
• Displaying all expert reviews and catalogue images on the selected clothes crowds the screen too much. Some elements’ orientation, scale, and location need to be refined.

By going in-store, we also realized more ideas to apply to our product, such as:

• Adding the price of items and the option to compare prices across merchants directly.
• Adding a Call a Friend option so shoppers can validate their choices with their friends.
• Adding an Ask Staff for Help button to help them locate items instead of navigating on their own, which might be useful for large stores with meandering hallways and/or messy categorization or hard-to-reach items

Designing and Prototyping Our Concept

We created app screens using Sketch and exported the necessary 2D assets into Torch. The final prototype was created at an actual retail store in Singapore, using clothes and mannequin that existed in-store.

While creating our prototype in Torch, we experienced some challenges and benefits of the app:

Pros of Torch

• No coding involved. Placing 2D assets in 3D space is as easy as sending the assets to the device (iPad/iPhone), and dragging them to the screen (the 3D space). To move assets in the space, simply drag them around the screen.
• 2D assets can be easily created and exported using tools that are familiar to designers, like Sketch/Figma.
• Interaction and preview of entire flow can be created and viewed directly from device (iPad/iPhone). Since the device also has a camera, this means that while creating the interactions we can also see and validate how they work in the actual environment/context (in this case, the retail store).

Cons of Torch

• Each time we started the app, we were required to reset the anchor point. This meant that setting up the prototype was very time consuming. It was difficult to find where exactly we placed the anchor the last time since there is no marker or indication of our previous anchors. It usually took us about 5 to 7minutes of trying repeatedly to find an anchor that would not make our prototype look too small or too large on the new set.
• Invisible layers show up as grey rectangles with no indication as to what those layers are when in visible mode.
• This is particularly troublesome since we often have 10 to 15 layers in one scene, and all of them are more or less stacked one on top of the other in space, and all are set as invisible at the start, except for the starting layer.
• After previewing, sometimes we needed to rescale or reposition an asset, but since that asset is set as invisible at the beginning, it was very difficult to find which one, among all the grey (invisible) assets, was the one we needed — especially if there are many neighboring assets of similar sizes.
• UI and flow for interactions is not as user friendly.
• There is no clear indication whether an object’s interaction is set to enabled or disabled.
• It is not intuitive what steps user needs to follow to create an interaction.
• Help and tutorial is not available from device itself. All tutorials and help need to be Googled on the Torch site.

What We Learned About Designing for 3D

As designers who are very familiar with 2D, working in 3D had its own unique challenges and benefits:

• Finding inspiration for ideation was a challenge. Most of the case studies we’ve looked at from IKEA, Google, Shopify, and Nike employed similar AR concepts: to display more information about the products, to navigate, to customize products, to visualize product in the user’s own space, or to educate. Most of the ideas we came up with also revolved around those concepts. It was difficult to break the mold and generate novel ideas. With limited references and case studies, we find that it is important to always ask ourselves, will this idea provide real value to users? Or is it simply for entertainment or novelty purposes?
• Some of the ideas we came up with had distinct 2D flow and feel to it, and the questions we asked ourselves most often to weed out these ideas were: Why should we do this in 3D with AR when it can work just as fine on screen with 2D, like a usual app? What unique benefits would AR provide through this concept?
• We often forgot in some ways that we’re no longer limited by the iPad/iPhone screen size. By having access to the entire 3D space around us, our designs may not be limited to the screen that we see. By moving our device around, we can utilize the entire 3D space around us.
• We can utilize the assets in real-world and are no longer limited through assets we need to design from scratch. As in our prototype, we utilized the mannequin and the clothing in store.

Conclusions

In some ways designing for 3D forces us to unlearn and adapt what we’ve learned in designing for 2D. From finding inspiration to creating the prototype, many of the mistakes and blockers we encountered stem from our attachment to the same, familiar mindset and frameworks. In Japanese Zen there is a term called “beginner’s mind” which is an idea that encourages us to not be constrained by what we already know. As artists, we realized that this is a useful habit to practice, recapture and take hold of throughout the years to cultivate our creativity.

Resources

• Retail trends: Vol. 2 — A guide to virtual and augmented reality for Australian retailers
• Brainstorming/Ideation: Mural
• Prototyping tools review: AR/VR Prototyping
• Prototyping: Torch App