Formal usability tests in a lab set-up are an excellent and proven ways to evaluate usability of a product in ‘use context’.

An hour in the life of a moderator

Running a test effectively involves planning for the session and devising a good test plan. The usability test protocol is a key element to run these sessions systematically and successfully. A typical protocol consists of two elements:

1. a script for the moderator to steer the test
2. a sequence of tasks to be performed by the user with cues for the moderator to observe and probe.

Typically, usability tests last anywhere between sixty to ninety minutes. The protocol thus tends to run into several pages and accessing it is often a hindrance while engaging in a dialogue with the participant.

Without quick reference material, the researcher typically relies on memory to steer the test in the pre-decided sequence. This is usually a considerable overload during a test sessions as the situation may unfold quite dynamically. A documented script and test plan is essential to remain orientated and execute task flow methodically.

An hour in the life of a participant

On the other side, the participant is briefed at the beginning about the structure of the session, but the session is largely in a blackbox, the only interface being the moderator. The participants rely largely on the verbal cues from the moderator at every step.

The insights from a usability evaluation session fundamentally rely on the observations and probes on the behalf of the moderator, it is thus essential that the moderator keeps the sessions light, engaging, exciting and motivating.

The Cue-Card method

To address some of these challenges, a a set of 22 visual cue cards were created to moderate long usability sessions for testing a new music application.

The sessions were structured in 3 parts:

1. Finding more about the music listening habits and behaviour of the participants
2. Usability Test to evaluate the new application
3. Semi-structured conversations to discover habits, needs and notions around personalising music

Eight cards were designed for session ‘A’ with the first card for the moderator to introduce and set the context for the study followed by questions to:

1. probe the role music plays in the life of the participant,
2. music listening habits on the phone
3. music applications in use on the phone
4. role of radio in their lives, and
5. behaviour and notions on mobile internet and more.

A set of 10 cards were designed to steer the session ‘B’ — Usability Test. Each cue card focussed on a single task inviting the participant to achieve a specified outcome using the music application. For some of these tasks ‘scenarios’ were curated to motivate the participants by situating them in a context. The cue cards illustrating the scenes visually were observed to stimulate the participants well.

And 4 cards were curated for the last session beginning with a ‘perception card’ exercise and steering semi structured interviews to probe specific points on their personal preferences to listen music.

Employing a new technique

Each card addresses one focal point in the test plan, thus becoming a communication vehicle. It offers a way of making concrete the task scenarios and enticing the participants to attempt the tasks. Some of the advantages experienced with the cue card method are :

• Takes away the overload of referring a long document (without making it visible to the participant) with a constant anxiousness of missing the points or sequence of tasks.
• Is a shared resource between the moderator and the participant thus making the interactions between the two more engaging, helping in deeper probes — essential to gathering insights.
• A visual cue keeps the participant motivated and in a playful spirit to perform the tasks.
• Takes away the monotony creating a narrative for each session in which participants feel that they have a role to play.

The test plan lays the foundation for the entire test and a script offers a better orientation for the act than an improv. “A diagram is a worth a ten thousand words,” perhaps it is worth attempting diagrammatic scripting approaches to construct visual outlines for conducting Usability Test Sessions.