Usability testing — understanding ‘real’ users to take products to the next level

Before we begin, let’s understand what usability is. The ISO definition says:

“The effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments.”

However, usability testing can also measure learnability — the user’s ability to operate the system at a predetermined level of competence after a predetermined period of time or after training.

Usability testing has been around for a long time and yet many companies, large and small, don’t factor it into their workflow. Reasons range from, “It’s too expensive, it’s not in our budget,” or, “We don’t need to do user testing, let’s just focus on the analytics as that tells us enough,” to “We don’t have the time or the resource.”

Let’s give it a try

If budget is the problem, let’s work around that. Using methods such as guerrilla testing could be a solution. This type of testing is cheap, flexible and simply requires one or two people to find a location where your users might be, then approach them and ask if they would be interested in trying out the new app or website. This is a quick and simple way of getting great feedback from users. It shouldn’t take a completely ‘wild’ approach, though— some structure should be in place to document users’ feedback.

Another approach is to start off small. Maybe only test a few users to begin with, offering a small incentive such as a £10 Amazon voucher. If you can’t bring them into the office, because you don’t have the space or the correct setup, no problem — you can still carry out tests remotely. All you need is a computer with Skype or Google hangouts, a voice or screen recorder and an internet connection. That’s it, and the same applies to the user.

You see, you can work around those ‘reasons’.

Why to run a usability test?

• It will help answer questions such as, “Can my users use the desktop interface, mobile app, website or web application, effectively, efficiently and satisfactorily?”
• It provides feedback on your product before the release when the risk is still low
• It allows designers to try different ideas without consequences of a live product
• It reduces opinion wars by upper management

Intention vs behaviour

Something I found interesting and very true was brought up during a workshop I attended recently. One reason we conduct tests is because intentions and actual behaviours are often inconsistent.

Let’s say I ask a user how many times they go to the gym every week and they reply saying, “About five times.” If I interview 100 users and they all say the same thing, I might use this information to help me design a gym that will be able to accommodate this amount of people.

In reality, most of those people might end up only coming twice a week. The gym that was built to accommodate a larger number of people could be left feeling empty with lots of unused areas and equipment.

We need to understand that people say what they ‘think’ they would do, but the reality is often different.

Usability testing can help us dig deeper into a user’s behaviour, especially when using techniques such as ethnographic research. This is where you monitor a user in their natural environment, such as their, home while they carry out a usability test. This type of testing is extremely insightful because users feel a lot more comfortable, which allows them to act normally, as they do in their day-to-day lives.

Common pitfalls

Testing the wrong users

Testing with unrepresentative participants wastes your time and invalidates your results. This can happen if you don’t screen the participants properly or you don’t identify the correct user groups to include in the testing — even testing with co-workers is not a good idea.

Trying to test too much

You can fit only so much into the typical session. If you try to include too much, you’ll run out of time or find yourself rushing through the sessions.

Acting on feedback from 1–2 users

One of the women sitting next to me during a recent workshop I attended brought up an interesting story about some testing they did on a medical app. The testing setup was pretty standard: there was a facilitator, note taker and observers (developers).

During one of the sessions, a user found it hard to use a certain feature on the app. One of the developers, who was observing, noticed this and hurried off to change the app. After making this change, the remainder of the users weren’t able to use this feature. When asked, the developer said, “Well, I saw that he battled to get the feature to work so thought I would quickly try and fix or improve it.” This was based on one user, whereas the best approach is to wait until a number of users have completed the test and then iterate.

Don’t make these same mistakes.

When done correctly, a user test can provide you with powerful knowledge and insight you can take back and share with your key stakeholders, allowing them to see the true benefits of speaking to your ‘real’ users. This type of information is vital if you want to create a successful product that will engage your users and win their hearts keeping them coming back for more, ultimately turning them into your loyal users.