6 Tips for Conducting Remote Moderated Usability Tests

Sharpen your user research skills to get better insights

Thanks to the increasingly sophisticated user testing tools on the market, remote user research is a growing trend that reflects the hybrid and remote work realities of today’s global workforce. At SAP, we’ve adopted user testing tools like UserZoom and virtual whiteboarding tools like FigJam and MURAL to incorporate our users’ and customers’ feedback and better shape our product experiences.

With remote usability testing solidifying as a new practice, I thought I would share our team’s experiences and learnings in the last few years when it comes to moderated remote testing.

Moderated versus unmoderated remote usability testing — What’s the difference?

Before diving into our learnings, it’s useful to distinguish between these two types of user testing:

• Remote moderated usability testing is similar to in-person moderated testing. The difference is that it takes place over video using remote-testing software.
• Remote unmoderated usability testing also happens over remote-testing software, but there is no moderator present during the test. The test participants complete the tasks on their own and the software captures the results.

Now that we have built a bit of the context, let’s take a look at some helpful tips that you can apply while conducting your next moderated remote study.

1. Always have your own camera on

Many test participants are first-timers and have anticipatory anxiety about who might be on the other end of the screen. As a moderator, this is a chance for you to welcome the participant with a smiling face. That means you should always turn on your video right from the start of the interview. This will set up the right tone and vibe for the rest of the session.When they see you — a normal human being, like them! — on the other side, it will relax them and bring down that stress level.
That being said, never push the participant to turn on their camera if they have not already done so, unless the research absolutely requires that the camera be on. Test participants may be camera-shy and it’s important to respect individual preferences.

2. Do a dry run before the big day

Rehearsing is crucial to ensure that everything goes to plan at the time of the actual usability session with the test participant. Invite a few stakeholders to conduct a dry run of the session as you would do on D-Day (the actual day of the session). This will help you get familiar with the video conferencing tool and its nuances. You could also set a timer and analyze how much time the interview is taking so that you can decide if you need to cut out some tasks.

3. Always have a fallback plan

During usability tests, we usually ask participants to interact with a screen and complete a number of tasks. To do that, we typically share the hyperlink of the testing environment. To prepare for the worst, always have a fallback plan ready in case your hyperlink is not working. For example, set up a duplicate environment or prepare a working prototype link and be ready to share it out immediately.

4. Skip job roles

There was an instance when I introduced myself as a member of the Design team to the test participant and halfway through the session, during a task when I asked some probing questions, the participant responded by saying, “I wouldn’t want to comment on anything about the design layout, but I feel it could have been better if……”.
What happened here? The participant felt uncomfortable about giving direct feedback about the design as he was aware members of the Design team were on the call. To avoid such scenarios, it is better not to mention our specific job roles to ensure that we are getting unbiased feedback.

5. Mute anyone who shouldn’t be talking

It is highly unlikely that you will moderate a session all alone. Typically there will be notetakers and observers with you on the call. As a moderator, make sure that you mute everyone else except the participant and yourself, as sounds such as keystrokes and breathing might be distracting to the participant.

6. Paste the tasks into the chat window

During usability testing, we usually share a prototype or a test environment and assign tasks to the test participants. While some tasks can be very straightforward, some might be more complex and involve multiple steps. Or when you say them aloud, the participant might not understand what’s being asked at first go due to a language or sound barrier. It is always a good practice to copy those tasks and paste them into the chat window as you articulate them, encouraging the participants to read the tasks out loud.

Nishant Bali co-leads the design team of Product Footprint Management at SAP. He has a keen interest in psychology and human-centered design. Nurturing up-and-coming designers in his spare time is his passion along with being an advocate of mental health and a pet lover.

Thank you to Marie-Simone Proksh for your input on this story.