The 4 questions to ask in a cognitive walkthrough
Although the cognitive walkthrough gets less coverage than Nielsen’s heuristic evaluation, it’s just as effective at uncovering interaction problems. It’s also an ideal way to identify problems that users will have when they first use an interface, without training.
About the cognitive walkthrough
The cognitive walkthrough is a formalised way of imagining people’s thoughts and actions when they use an interface for the first time. Walkthroughs identify problems that new users will have when they first use an interface. You select one of the tasks that the design is intended to support and then you step through the task, action by action, seeing if you can identify any problems with the interface.
Although the technique was developed over 20 years ago (by Cathleen Wharton, John Rieman, Clayton Lewis and Peter Polson) it is much less widely used than heuristic-based expert reviews. This is a shame because the technique simulates the way real people use an interface: by exploration rather than by reading the manual.
Creating the happy path
Before you can start a cognitive walkthrough, you need a complete, written list of the actions needed to complete the task with the interface — the ‘happy path’. For example, here’s the happy path for creating a customised voicemail message on an iPhone:
- Tap the ‘Phone’ icon
- Tap Voicemail.
- Tap Greeting.
- Tap Custom.
- Tap Record and speak your greeting.
- When you finish, tap Stop.
- To listen to your greeting, tap Play.
- To re-record, repeat steps 4 and 5.
- Tap Save.
Sometimes, creating the happy path is all you need to do to realise there is a problem with the interface. In a classic example from an early smartphone, configuring email required 78 separate steps. If your happy path has this many actions, there’s no need to continue with the review: you’ve found a serious problem already.
Once you have the happy path, you’re ready to start the walkthrough.
The cognitive walkthrough is structured around 4 questions that you ask of every step in the task. You ask these questions before, during and after each step in the happy path. If you find a problem, you make a note and then move on to the next step of the task.
Q1: Will the customer realistically be trying to do this action?
This question finds problems with interfaces that make unrealistic assumptions about the level of knowledge or experience that users have. It also finds problems with systems where users expect to do a different action because of their experience with other interfaces or with life generally.
For example, my Sony Vaio laptop has two settings, ‘Stamina’ and ‘Speed’ (see Figure 1). The designers assume that I will switch to the ‘Stamina’ setting to save battery power and use the ‘Speed’ setting for graphics-intensive applications, such as games. Is this assumption reasonable?
Q2: Is the control for the action visible?
This question identifies problems with hidden controls, like gestures on a tablet or a phone where it’s not always obvious what you can do. It also highlights issues with context-sensitive menus or controls buried too deep within a navigation system. If the control for the action is non-standard or unintuitive then it will identify those as well.
The world of TV remote controls provides a familiar example. Remote controls often contain a flap to hide features that are rarely used (see Figure 2). The problem occurs when you need access to those functions but, because you rarely use them, you don’t realise you need to lift a flap to reveal them.
Q3: Is there a strong link between the control and the action?
This question highlights problems with ambiguous or jargon terms, or with other controls that look like a better choice. It also finds problems with actions that are physically difficult to execute, such as when you need to press three keys on the keyboard at the same time (and stand on one leg).
Figure 3 shows an example from a car park machine at Stuttgart airport that has a weak link between the control and the action. I used this machine over a period of 18 months while on assignment in Germany. The first time I arrived at the airport and wanted to leave the car park, I was confronted by the barrier and the control shown in the picture. The barrier has a slot at the top and two buttons, a green one on top and a red one below. I didn’t have anything to put in the slot, so I guessed I had to press one of the buttons.
Question: Which button would you press to lift the barrier: the green one or the red one?
Convention dictates that you would press the green, upper button. In fact, this appeared to do nothing. Assuming that the barrier was broken, I pressed the red button, thinking that this would put me in intercom contact with the car park attendant who could then open it for me. To my surprise, the red button lifted the barrier.
Clearly, I was not the only person to experience this difficulty. When I returned some weeks later, the design had been upgraded (see Figure 4). As well as a large sign showing the correct button to push, some ‘help text’ had been added to the system (in both German and English) saying ‘Please press the red button’. To emphasise this, the designers had even printed the word ‘red’ in red ink. Which button would you press now?
As if to prove that customers do not read documentation, when I returned some weeks later, the design had been changed again (see Figure 5).
Presumably, customers had not been using the help system. The hint text, now flapping in the breeze, had been almost discarded and the design had been changed to include a series of (red) arrows, indicating which button to press.
This month’s rhetorical question: how many participants would have been needed in a usability test to spot this blooper?
Q4: Is feedback appropriate?
This question helps you find problems when feedback is missing, or easy to miss, or too brief, poorly worded, inappropriate or ambiguous. For example, does the system prompt users to take the next step in the task?
Figure 4 shows an example from a control panel for an electronic toilet door on a British train. What button would you press if you want some privacy?
With this system, you first need to press the ‘Door open / close’ button and then you need to press the ‘Door lock’ button. If, like many people, you forget to press the ‘Door lock’ button, you may find your privacy interrupted as someone on the other side of the door opens it to reveal you seated on the toilet like a prize in a game show. This problem occurs because the door fails to provide adequate feedback on whether it is locked or unlocked.
Try it yourself
The best way to get going with a cognitive walkthrough is to try it yourself. Start off by writing down the happy path for sending a message on your phone. Then walkthrough each step in the process and ask the 4 questions of the interface.
Why not join the thousands of other people taking my free online user experience lessons?
Originally published at www.userfocus.co.uk.