User Experience Design Principles for Mobile Augmented Reality Applications
Augmented Reality (AR), in which interactive virtual objects are seamlessly superimposed into real-world environments, is a growing area in mobile applications. With the advancement of mobile devices capable of producing increasingly more persuasive compositions, the vast potential of Mobile Augmented Reality (MAR) has begun to be explored. To ensure the acceptance and success of future MAR systems and fill the gap in MAR research, it is vital to identify current UX guidelines that apply to MAR applications. This paper identifies existing UX design trends in MAR and surveys current MAR applications to discuss the potential areas of improvement.
KEYWORDS
Mobile Augmented Reality, Design Guidelines, User Experience, Survey.
1. INTRODUCTION
As computer technology increase in power and decrease in size, and hardware-intensive applications are rapidly becoming feasible, mobile phones have become increasingly prevalent [7]. This evolution and flexibility of mobile phones have created a perfect platform for Mobile Augmented Reality (MAR) [7] applications, utilizing Augmented Reality as a unique feature to produce an enhanced composite view of a user’s surroundings by mixing real and digital information. While MAR applications have continued to gather intrigue and growth in development, a major concern is how user perceptions and experiences can be significantly limited by the design of the User Experience (UX). Since much of the previous work has been limited to AR technologies, research on MAR UI and UX design remain mostly unexplored. To overcome the limitations of MAR and addressing the gap in MAR design research, this paper examines current MAR applications in different domains to determine the current user experience design patterns. Using previous research in AR as a starting point, this paper will first introduce the basic elements of user experience design such as environmental design, interaction design, and the utilization of visual cues. Then, a survey is conducted with the help of 5 volunteers using 10 noteworthy mobile apps in four distinct categories: in education, business, entertainment and social. Finally, the gathered data is used to discuss the effectiveness of the current MAR application design and identify areas of improvement.
2. UX DESIGN PRINCIPLES FOR MAR APPLICATIONS
2.1 Environmental Design
Due to the intrinsic reality-based nature of AR in Mobile applications, developers and designers must consider the context of the end user, which has three fundamental aspects: physical scenarios, spatial space, and accessibility [4].
2.1.1 Physical Scenario
A user’s physical scenario indicates his/her usage when operating an AR application. UX designers must define user journeys and consider how the interface will react in each physical scenario. Rob Manson [4] from AR UX lists key examples of AR usage: Public, using the whole body to interact with the software; Personal, using a device in a public space; Intimate, sitting or standing with minor movement; and Private, using a wearable. It is vital to consider these physical scenarios [4] as it can place certain limitation on application functionality and design choices. Certain applications may require initial calibration or setup before the application can be run.
2.1.2 Spatial Space
Considering a user’s spatial space involves examining the lighting, sound, and motion of a user’s immediate surroundings [3]. In all cases of augmented reality applications, moderate ambient light is required for the camera to detect real-world elements. Harsh glare from sun outdoors or extreme contrast between shadows and sunny areas indoors may interfere with the sensor’s ability to detect real-world elements, especially when using planer detection [10] [9]. Similarly, ambient background noise is another concern if an application needs to generate adept volume to be heard or detect speech or other sound inputs [9]. Developers should accommodate for potential sound and light sources that would impact the system’s fundamental abilities. Moreover, the environmental design should factor the motion of the user. In MAR applications where the user is required to maneuver in the real environment, spatial awareness and as well as safety needs to be addressed. Since much of the user’s focus attends to the flat 2D screen of the MAR device, users may become unaware of their surrounding environment and potential elements of obstruction or hazard. There’s a variety of precautionary alerts and sensory feedback that can be implemented to ensure the safety of the user. Issues related to using a 2D display in integrating 3D AR technology and practical solutions is discussed in further detail later in Section 2.3 Visual Cues.
2.1.3 Accessibility [3] [9]
To strengthen the usability of MAR applications, flexible adjustments should be allowed for user preferences and ease of use. Here are some factors to be considered: user engagement time, one-hand control in-app navigation, time restrictions to prevent unintentional operations, effective audio cues for the user’s who are unable to see and audio detection.
2.2 Interaction Design
An effective Interaction Design improves all aspects of user-system interaction, especially application navigation. Since most user interactions is spent on operating the application, the user interface should easy to understand and use. A common technique to improve usability is to place the control buttons on frequently used and easy to reach areas of a device’s touch display. Applying haptic feedback in MAR application is another tool that can be used to provide helpful information and reduce confusion [10]. Another aspect that influences a user’s mindset and experience is the design of user journey maps [6]. A successful journey map design should effectively organize the flow of information presented on the screen, ultimately assisting users to accomplish tasks as fast as possible. Efficient journey maps not only reduce the cognitive load on the user but also reduces learnability [6]. The UX design must also compliment the application’s purpose [8]. For instance, the utilization of exciting AR filters and the design of the unique sharing platform in Snapchat enables users to better the user’s social interactions. For applications that require users to focus on the image produced by the camera, a reticle or raycast from the device may be useful [3]. Moreover, designers and developers may also scale, or animate digital elements based on the orientation and position of the camera [10]. Having virtual characters react with user movement, or revealing more information about an object as a user approaches, are some proximity cues [3] that encourages exploration and interaction via movement. The most exciting factor of MAR is the ability to capture and transmit a user’s augmented view allowing for unique user communication opportunities. A user can draw, add or manipulate virtual elements in the real- world context captured from another user located elsewhere. While the technology in this area is still underdeveloped, some developers have already conceptualized this idea, in Holo for example [10]. For effective user interactions, the terminology used should be consistent and understandable. Although there is currently no standardized terminology for MAR applications, a baseline of interoperability of terminology between MAR applications should be established [9]. While there are some terms used in traditional mobile application also apply in MAR applications, newer terms that are associated with unique AR aspects may become confusing if a standard is not set. Development of vocabulary along with different standard interaction gestures in MAR needs to be considered as a form of interaction design so that the widespread adoption of these terms would allow users to expect [10] [9].
2.3 Visual Cues
For users to experience the true potential of AR, it is essential for designers to apply visual cues to improve the presence of virtual objects and encourage users to explore their surroundings. There are multiple methods for enhancing an object’s appearance: increasing an object’s weight, utilize depths maps, and choose different material properties. Designers can also consider projecting virtual objects with specific choices of lighting techniques or casting shadows to improve overall object depth and presence. A common lighting model is simply placing a light source overhead at the 12 o’clock position, parallel to the device [2]. In addition, AR development platforms such as Google’s ARCore allows devices to detect varying plane sizes or overlapping planes at different elevations [3] allowing for unique volumetric responsive designs and allows creators to better react to the constraints of users’ mobile play space. That said, designers must be aware of applying visual cues that force fixed paths on users or limit’s a user’s view [10]. While visual cues are essential in areas of projecting virtual imagery, visual cues also enhance UI elements and provide clues for better interaction. Similar to current mobile applications, UI elements such as icons, buttons, menus and text boxes must accommodate the application’s purpose and function [5]. Developers should design UI elements to improve discoverability and user efficiency. To make buttons and other interactable elements more easily identifiable, hover states can be added to highlight objects of interest [10] [3]. Similarly, UI elements that manipulate AR projected objects should give active visual cues in forms of realistic feedback. For instance, in a furniture shopping application, the illusion of gravitational force can be emphasized when users drag and place a bookshelf [9] [3]. Visual cues are also tools to differential the appearance of UI from the user’s surroundings. Principles such as the colour theory, typography, opacity, and drop shadows, are just some of the rudimentary considerations for practical and usable UI design. [2][5][6].
3. DISCUSSION
To understand the UX of mobile AR applications in the current landscape, I selected ten apps from different disciplines of retail, entertainment, productivity and social to be evaluated in a survey. Next, five participants were selectively chosen to understand varied perspectives; applicants belonged from networking, gaming, design, neuroscience and engineering backgrounds. For this exploration of UX design in existing MAR applications, the survey was made using “The Questionnaire for User Interaction Satisfaction (QUIS)” which is a tool developed by the Human-Computer Interaction Lab (HCIL) at the University of Maryland [11]. The QUIS is a highly reliable measure, designed to assess users’ subjective satisfaction across different human-computer interfaces. Figure 1 shows the evaluation of UX design for MAR application.
The following are the findings from the analysis of the applications:
- 80% of the apps are easy to use and are quite flexible. Even though the apps have good quality, they are either mostly satisfying or not stimulating enough. Potential improvements can be made in terms of social engagement, such as in the World Brush app, where users can create unique drawings that are anonymous but lacks tagging or sharing abilities.
- The information in 90% of the applications is readable but fails to structure and direct the flow of information presented. In AR Runner, for example, is a game offers different modes of play but lacks instruction on how to play select or switch the differing play options. It is only after playing the application a few times, does the user understands the flow.
- 10% of the MAR applications still face some issues with operation speed, especially if dependent on the internet network. Applications, such as Google Translate, should offer offline assistance or operate with downloaded local files.
- Users mark 30% of the applications require clearer input prompts. For instance, Graphmented, which creates charts and dashboards using AR, have buttons for control settings but are not highlighted enough for participants.
- 80% of the applications feel that a tutorial or additional instruction is needed. This suggests that there does exist a learning curve in some MAR application. In Inkhunter, where users can try on virtual tattoos, only show a set of information only once at the beginning and does not show any additional technical support when participants struggle with the task at hand. Inkhunter also increases the cognitive load and stress as it is harder to use the application with one hand.
Applications with a functionality use case like Google Translate, Chalk and Seek were given more preference over other business or entertainment disciplines since they can increase efficiency. Most applications proved to be reliable except ‘Chalk’ which requested access to contact details of the users arising some privacy issues for some users.
Lastly, 70% of the users feel the visual system of applications needs further improvement and more active engagement. User experience and interaction was hindered in Houzz, due to the absence of weight and depth in the virtual objects, and in Google Translate, where the interface is too conservative and rigid.
It is evident from the data gathered by the survey that there are several areas for improvements in MAR applications. The analysis shows more than 60% of users are willing to use these applications in future and 50% would recommend it to others. The overall rating of the experiences of AR is above average for more than 60% of users. AR proves to be a stronger tool where the applications have functional usage. It seems current MAR applications still use traditional guidelines for mobile applications and thus suffer in certain areas involving AR. This brief discussion outlines the potential opportunities for AR applications and how these gaps can be addressed with the ever-evolving AR technology.
5. CONCLUSION
Recent advances in mobile augmented reality is creating an exciting design space for new AR applications. While the UX community is still finding ways to participate in shaping this design probe into the experience of everyware, we still lack fundamental knowledge regarding how various design decisions will impact AR application effectiveness. This research analyzed existing user experience design guidelines and organized them into specific categories of environmental design, interaction design, and visual cues. Using the literature review of the UX design, ten MAR applications were evaluated to explore the challenges and opportunities in the current AR design scenario. While our experiment only assessed a subset of MAR applications, the survey framework was enough to examine the current trend in MAR industry and how these applications can be improved. Overall, the consensus demonstrates the need to enhance UX design for AR applications and the importance of considering design through the use of these guidelines.
The biggest misconception is we look MAR as a single user interface, and from that perspective, it doesn’t add to what we already have. AR becomes strong when we look at situations where we have a high granularity of technology in the space where a standard desktop could lose control of these systems. We need to aim to improve the technology that is open and is owned and accessed by everyone.
REFERENCES
Book
[10] Michael Haller, Mark Billinghurst, Bruce Thomas, 2018. Emerging Technologies of Augmented Reality: Interfaces and Design. IGI Global. January 01, 0001.
Journal
[5] Patricia Search, 2016. Information Design Opportunities with augmented reality application
[9] Koelle, Marion, Patrick Lindemann, Tobias Stockinger, and Matthias Kranz, May 2014.Human computer Interaction with Augmented Reality.
Conference paper or contribution volume
[8] UXmatters. Accessed March 16, 2018. Inside Out: Interaction Design for Augmented Reality.
[7] Zugara. May 02, 2016. Augmented Reality and Virtual Reality Accelerating Evolution Of User Interface Interaction. [11] the University of Maryland. Accessed April 01, 2018. Questionnaire For User Interaction Satisfaction.
[6] Google. Accessed March 20, 2018. Principles of Mobile App Design: Engage Users and Drive Conversions.
[4] Webdesigner Depot. July 17, 2017. Accessed March 20, 2018. The Essential Guide to UX for AR.
[3] Unpingco, Alesha. December 13, 2017. Accessed March 20, 2018. Best Practices for Mobile AR Design. Google.
[2] Wilson, Tyler. November 13, 2017. Accessed March 16, 2018. The Principles of Good UX for Augmented Reality — UX Collective. [1] Skjoldbroder. August 11, 2017. Accessed March 16, 2018.Prototypr.
