Usability testing 101

By Christopher Ellis (Digital Product Designer)

At Mentally Friendly, testing is at our core.

It taught me how to let go of any creative ego, which has little space to breath when your work is put through its paces in real world scenarios — it keeps everyone honest and ensures our ideas are being validated throughout the process.

The Spy Room™

It’s amazing because it:

  • Brings up roadblocks
  • Removes team bias
  • Allows pivoting easily
  • Moves the team from thinking to knowing

All in the shortest possible timespan.

What is usability testing?

A usability test consists of gathering unbiased participants and asking them to perform specific tasks on an interface we created.

An interface can range to a paper prototype to test a navigation structure, or a checkout form with live data scraping from the clients server.

It is to ensure that something works, to find out how self explanatory things are and if a person of average ability or experience can use it.

What it isn’t

A focus group

Why is it important?

When solving complex problems it is always good to get validation early so you can course correct if need be.

On exisiting products it can identify bottleneck points and on a new product it allows the product team to gain fresh eyes who have not been on the same journey we have.

What do we test?

We focus on testing functionality, rather than design. It is not a method to crowdfunded design ideas.

Avoid crowdfunded design

When do we test?

There is no time like the present — aim for less participants and more frequency.

Testing can occur when you:

  • Already have an exisiting product
  • You have a new idea
  • You are just starting to build one

It is not something to cram into the end of a project so you can tick the box that you did it. It is meant to be part of an ongoing iterative process, with the underlying goal of improving the product.

Types of testing

1. Usability

Generally speaking this is the “hallway testing” recruitment method. We are testing the usability of the interface — the main focus of this article.

2. Concept

This is where you recruit industry experts, testing initial reactions to a new concept.

Who do we test on?

For usability there is usually no needs for a target audience, if someone with an average digital skill set can’t perform the task, the UX needs to be improved.

For both avoid choosing someone with a design or UX background as they immediately turn from being test subjects into being “expert reviewers”

What do we look at?

We Create a series of testing tasks scenarios — real life mini ‘challenges’ for the user to try and complete using a website or app.

Make it crystal clear on what needs to be done and don’t lead the subject.

Instead of asking “Can you find the settings page?” perhaps phrase it “You just got a new phone number and wanted to update it in the system, how would you do that?”

We really want to know:

  1. Can they complete the tasks unassisted?
  2. Are there any blocks preventing the user from completing the tasks?
  3. Are they completing the task as we intended? If not how are they navigating to complete the challenge.

How to create a scenarios

Have a goal in mind (find the best price) → anticipate user steps (find and use a special offer) → take them through obstacles/cross roads (deal applicable to a portion of a shopping cart)

Considering hypothesis:

  1. What is the objective of the website? What are the overarching goals that we want the user to complete?
  2. Since the last sprint, what are the major updates that have been added to the UX?
  3. Are there any ‘new concepts’ on the website that might not be obvious to the average user?

Have there been any considerable design challenges during this sprint?

Usability testing session in progress

Our Tools for Testing

The Spy Room™

Our Observation room, colloquially know in the studio as The Spy Room™ has cinema styled seating, 5 viewing monitors and the air conditioning to match.. oh and a two way mirror.

This is where the team can witness the interactions in an unobtrusive way and allows the team to take notes and discuss outcomes in real time.

It also relieves pressure on the tester to take notes as they should be focused on running the test as they may miss important insights. is a tool created a team of ex-Spotify engineers that allows teams to record onscreen activity within mobile apps and websites.

It even record the user’s face and voice, so they can explain what they’re doing when they encounter a bug or some other problem.

We found lookback is great for reviewing sessions and also sharing to stakeholders who can not attend.

Top tips

  1. Keep overarching goals at the core of the process
  2. Test functionality, not design
  3. Prioritise frequency of testing over a number of participants
  4. Test the product, not the user
  5. Try to make them feel as comfortable as possible
  6. Avoid leading people
  7. Stimulate thinking out loud
  8. Write down insights throughout the test
  9. Have people watching to take notes
  10. Any test is better than no test

This article was based on a deck prepared by Tadas Lukosevicius (Digital strategist)

Find me on Twitter

Enjoyed this article? Here’s another.

Like what you read? Give Mentally Friendly a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.