Contextual Usability Testing
Considering the Context of Use While Designing, Prototyping and Testing New User Experiences
Context strongly influences how we use and interact with digital products and services. Technology enables us to consume such services anywhere and anytime. Nevertheless, as users, we expect the interactions to be frictionless, direct, and instantaneous. Hence, as UX designers and experience architects, our designs, prototypes, and usability tests should take the context of use into account.
At adorsys, we explore and investigate human factors in the interaction with Conversational Interfaces. One of our areas of research is finance and banking. Specifically, we concentrate on designing and exploring services which offer new approaches to personal finance management (PFM). Additionally, we proactively take advantage of the possibilities afforded by the modalities provided by multi-modal Conversational interfaces and cross device applications.
Conversational Banking is qualitative banking made accessible.
Context of Use as Factor in the Design of Experiences
“Today, users’ activities are broken up into small bits of action strewn across time, space, location, and devices.”
Holtzblatt (2016), Contextual Design, Introduction
Context is a decisive factor when designing interactions for digital services. In the Contextual Design process, Karen Holtzblatt and Hugh Beyer emphasize the importance of context and situational constraints for designing user experiences. These are extremely important when designing for mobility, like for mobile apps or for Conversational Interfaces. Experiences, for which the computing device becomes contextual, or even ubiquitous, are strongly affected and shaped by the richness of the context. When testing assumptions for such applications, we cannot defer the context as if it was a collection of neglectable supplementary aspects.
“We must design our target activity so it fits into life on the go: the unstoppable momentum of life.”
Holtzblatt (2016), Contextual Design, Introduction, on Design For Life
Contextual Prototyping for Conversational Banking
When evaluating the usability for specific scenarios, obtaining a concise perspective on the context is critical. Although, Conversational Interfaces really excel when we want to provide qualitative information, in certain contexts the usability can benefit greatly.
Initiating payments is one of the core concerns of financial services. PayPal, one-click purchases and Apple Pay — initiating payments has to be seamless and instantaneous. Having to sit at our computer or to go to a bank branch for initiating a payment feels old-fashioned at best. We consume anywhere and anytime — thus, we expect to make payments correspondingly.
“As consumer expectations change, there is growing demand for frictionless interactions that span the digital and physical worlds[…] The age of connected commerce is here, and voice-activated smart devices will play a pivotal role in the future of payments by streamlining the way consumers make purchases every day.”
Devin McGranahan, Senior Group President, Global Business Solutions at Fiserv
Besides considering a frictionless experience, payments also have to be secure. We cannot compromise security in order to offer a more polished experience. But how can we conveniently authorize the access to the account for initiating payments?
The members of the Conversational User Interfaces (CUI) team at adorsys believe in and explore user-centered, contemporary banking experiences, like initiating payments via voice. To make online banking processes more secure, we are habituated to dealing with multi-device workflows for commanding and authorizing access to our bank accounts. Using a biometric sensor of our smartphone as the second factor for authorization is quite common.
Living Room Scenario — Daily Banking
Naturally, we designed our voice-banking services accordingly. While interacting with a Voice-User Interface (VUI, like Amazon Echo or Google Home/Google Nest mini) for retrieving information and initiating commands, the system integrates the users’ smartphones for authorizing access to critical data. A multi-device workflow allows to achieve a higher level of privacy than using a password or PIN via the VUI. Additionally, this approach offers convenience and speed.
However, this multi-device workflow inherently carries the risk of interrupting the flow and infusing friction. Users have to grab their smartphones — and it is not unlikely that they have to locate it first. When they got their phone, they have to use it for authorization and continue the interaction via the VUI. It is important to note, to guarantee privacy, VUIs shut a conversation normally down after a couple of seconds. Therefore, the users have to continuously engage to keep the channel open and the conversation alive.
All these points sound like ingredients for a recipe leading to friction and interruptions. Thus, we were eager to evaluate this workflow with some test users. What are users thinking in those moments? Do the assumptions we based our design on fit their mental models? Of course, we could have asked our testers to take a seat at a table, have the voice assistant in front of them, the smartphone next to them, …. However, of all the situations we thought of, such a setup seemed the utmost unlikely to us. Hence, if we had tested in such a setup we might have gained significantly less insights.
Instead, we prototyped a situation one might actually encounter in one’s living room. Our test users were asked to take a seat on our cozy sofa with a side table in front. We positioned the smart speaker at a distance of about 1.5 meters on a table. A bit further away, we placed the smartphone, which needed for the authorization, in the pocket of a jacket hanging at a clothes rack. The test users received a voice message from their daughters, via the voice assistant. The daughter asked, if she could transfer 50 € as soon as possible. Of course, our thoughtful voice assistant immediately proposed to start our Voice Banking application.
Hence, we created a situation which embodied our testers in a context closer to real-life situations. They sat on a sofa and suddenly the need to initiate a payment emerged. With our test environment we wanted to answer specific questions: How does our design guide the users through this interaction involving multiple devices? Do we provide enough and useful impulses so that the users keep the channel with the voice assistant open? What are the phrases the users use to keep this channel active?
Our setup and our prototyping toolchain help us to obtain answers to these questions. This was only possible by taking the context into account. Our toolchain enabled us to modify and adapt our prototype quickly. We ran a couple of pre-tests shortly before starting the ‘official’ sessions and, in no time, made smaller changes to our prototype. Thus, the sessions already started with the improved version.
Although, we designed our conversations carefully to take Clean Language into account, the pre-tests revealed some opportunities to improve the conversations by slightly fine-tuning the formulations. We used the re-prompts (we simulated a system with a single re-prompt in our prototype) to keep the channel open — asking, if the user was ready to continue. We got valuable input during the pre-tests — fine-tuning this conversational turn iteratively.
No matter how convenient and ubiquitous banking from the living room becomes, there are far more and more divers places from which people want and need to initiate payments. Exxon Mobile, Amazon and Fiserv just presented their take on paying for fuel at the gas station (for more information).
In-Car Scenario — Initiating Payments While Driving
Of course, Voice Banking and making payments via voice get even more interesting if we consider situations in which we are facing ‘situational impairment’, or some form of constraint, or restriction — like when we are driving, or riding a bike.
Depending on the context (e.g. in-car) there are different means for identifying users conveniently — we can make assumptions based on the car’s owner, the owner of the smartphone connected to the car, or via some form of voice recognition (Voice ID, etc.). Therefore, we decided to explicitly distinguish between identification and authorization, and to specifically focus on the authorization process.
Needless to say, testing the usability of a payment process while driving made us curious. While driving, hands should be on the steering wheel and the eyes on the road.
How can we test such a workflow and learn something about its cognitive load? Taking a prototype on a ride in a pool car might provide some insights and answer, however, this is clearly not feasible. Neither is organizing a professional driving simulator for early stage prototyping.
Furthermore, we considered using a commercially available car simulation game for PC or Playstation. However, the level of control such a game provides to set up our scenario is very restricted. Howbeit, we decided to develop a proprietary driving simulation — and by iconifying and stylizing, allowing us to set focus on driving and making payments.
Our simulation environment has to help us to generate the cognitive load of driving and to estimate the task load of initiating a payment concurrently. We aim for a comprehensive and familiar traffic situation: The persons have to drive on a three-way route and avoid crashing into other cars while passing them, and to leave the road on either side. At the same time, they are asked to buy and pay for a toll ticket before crossing the border to Austria.
Our prototype establishes important constraints encountered in such a situation. The testers have to use cognitive processes equivalent to those in real life. We try to evaluate the task load by observing the testers and by self-assessment of the testers (NASA TLX).
Bringing Context to Prototyping and Early Usability Testing
“[Prototypes are] immensely useful, as they’re all about learning fast and cheap.”
Cagan (2018), Inspired, Chapter 8
Of course, there are countless usability concerns, which need rigorous testing under lab conditions. However, the context and situational awareness is of ever-increasing importance. The great variety of conditions encountered in the context of use have to be investigated and taken into account already during ideation and discovery. Prototypes, which enable situation-aware design and testing, lead to a deeper understanding of the targeted users and support the ideation and discovery process.
A German version of this article can be found on ‘Der Bank Blog’.
References
Cagan, M. (2018): Inspired — How to Create Tech Products Customers Love, 2nd Edition, Wiley
Holtzblatt, K., Beyer, H. (2016): Contextual Design: Design for Life (Interactive Technologies), 2nd Edition, Morgan Kaufmann
McGranahan, D. according to: VoiceBot.ai, Bloomberg (last accessed on January 8, 2020)
