Remote Usability Testing during Covid-19: A New Opportunity

This year has caught us all by surprise. For the first time, in almost 100 years, all countries around the world are united and fighting against the same issue: coronavirus.

Digital experience has jumped up at the top of the agenda for businesses and companies, and our role as designers is to guarantee that users can fulfill their needs in this delicate world context.

This new (and stressful) lifestyle has given us the opportunity to research new methods regarding usability, accessibility, and business transformation to not only generate meaningful and innovative digital products but also sustainable businesses.

The Importance of Usability Testing in Digital Experiences

Before launching and having daily sessions of internal design iteration, we need the most important validation: User Point of View.

“Even the best UX designers can’t design a perfect — or even good enough — user experience without iterative design driven by observations of real users and of their interactions with the design. Usability testing helps us to uncover problems, discover opportunities, and learn about users.” Nielsen Norman

At Wolox, we always execute our usability testing sessions between our low and high fidelity prototype. This helps us to test in an early stage of our projects and allows us to apply our findings in the UI stage. Moreover, it provides us with the time needed to test against any iteration before delivering to the development stage.

We allocate almost two weeks for: recruiting the right users, planning the hypothesis we want to test, executing all of the tests (we usually test 5 users in total based on Norman Nielsen testing theory), analyzing the findings, and designing a report to discuss our action plan. Before the lockdown, we used to conduct remote usability sessions with tools, such as Lookback or UserTesting, which allowed users to take the test by themselves while we, as UX designers, could observe their behaviour. But, in my opinion, the best results are obtained when you are able to perform those tests with the users onsight. In this way you can ask any questions that may arise during the session and you can almost observe the user’s context while interacting with your designs.

This new context has brought us the chance to combine these two experiences in remote testing while being online with the user. As a result of having the opportunity to conduct, plan, and moderate 12 remote user testings in the Covid context I have found some challenges and new opportunities that I would love to share:

1. Remote can also mean Flexible

While planning your hypothesis and tasks, take into consideration that users might be conducting the test from home. Bear in mind that the time where you test might be considered as an additional metric to analyse. For example, if I’m conducting an usability test for an ecommerce site and web analytics show that users usually shop during lunch hours, this could be a variable to test in order to understand customer and user behaviour. Note that the time set of the usability test might be a conditional of analysing and testing.

2. The results are more reliable

Covid has forced us to stay more time at home, and as UXers this is a new opportunity. Our test results will be even more accurate if users perform the actions in their own real context: their own computers, houses, and where they usually perform these type of tasks. Thus, our reporting and metrics will be even more accurate. It allows us to test even more user’s behaviours such as interactions with their own devices or movements and actions they may need to do. For example, taking more time because they look for other stuff (web searching or a personal ID, for example) that may help them perform to their tasks. This will have an impact on what is being tested and the impact this feature may have in the future. We will obtain more accurate results because the user is already performing the tasks where we expect them to use the future product.

Of course, testing when the users are at home could be a disadvantage if the context to perform the task is different, for example, a grocery store.

3. Getting the most of your tools: Pros and Cons of Lookback & Marvel

In order to perform well the tests, it is important to consider the right tools to gather the information you need. We know for a fact that usability testings have to be recorded. As we are in remote sessions, we need the users to have their cameras on so we can capture the overall experience and their face expressions. There are lots of usability tools that have camera features. Here are my favourites:

Lookback:

• Pros: you can perform the whole experience in the platform. Also, you can invite members to play different roles. The observer have features like marking milestones that occur during the session and the user has a camera and screen recording to have a complete video session.
• Cons: the user has to install a plug-in on their desktop which can delay or interrupt the session. This can be easily solved by asking them before the test if they could install it previously. NOTE: the plug-in is necessary to perform the test.

Marvel Usability Testing

• Pros: If you have your prototypes in Marvel you can link those prototypes and generate a usability session. Marvel allows you to record the user with a camera + screen + voice. It also gives you metrics to analyze time + navigations.
• Cons: Metrics might not be accurate; Marvel only records during the time the user changes screens, so if you stopped and asked questions or the user made some comments , this may affect the metrics Marvel is tracking.

Here is my secrete tools recipe:

In order to have the most amount of resources, I conducted the tests using Google Meets, so we can involve an Observer in the session, and recorded the sessions with Quicktime (screen) + Marvel (to have the metrics) and always asking the user to share their screen after the prototype link is shared.

4. Tips on Icebreakers for Usabiliity Testing

It doesn’t matter if we are in different time zones, countries or regions, as moderators we always have to make the user feel comfortable. That’s why Icebreakers to break virtual tension are very important.

Start by :

• Always, asking how this particular year has been; This will generate an immediate bond because, for the first time, we all understand what we are going through.
• Ask questions you would ask to your user persona, regarding their behaviours, pains, goals etc.
• And lastly, and to create even more bonding, if the user has their video camera on, you should also have it on.

Wrapping up

Usability testing is one of many ways of gathering user findings and behaviour over digital experiences. This methodology brings us, UX designers, the opportunity to always try, experiment and iterate. This new world context has taught us that we can adapt by letting new tools arise to make our lives easier.