5 Steps to Better User Testing

Perfecting the user testing process can help you identify key areas for development in your re-design or when creating a new site or application. However, thorough planning isn’t necessarily the gold standard for good UXD. Too often, planners and designers place a lot of emphasis on creating lengthy documents that cover a wide variety of areas but fail to identify biases or to properly use these insights to inform the design process.

Ultimately, this makes it all a bit redundant and can leave large, and potentially costly, gaps in execution. The best way to get the most of your planning is to put in the hard work but to make sure you’re considering the most integral points. Let’s explore a few ways in which you can dramatically improve your user testing for better results.

Step 1. Understand User Bias

Pre-screen your candidates to try and establish and eliminate biases. Ask potential users a set of pre-defined questions to help bring these to light. Cover frequency of use, general online behaviour and their perception of the current service (if the revamp is based on an existing service). For greenfields projects, try to get a sense of what irritates them on similar or competitor applications. In instances of extreme bias, this user may not be right for this test, however, smaller biases can make them eligible (considering that everyone’s biased in some way). Still, take these into account when evaluating and implementing feedback.

Step 2. Vet Users

Aside from understanding bias, other forms of vetting are necessary. Ask your potential tester questions that will determine whether or not that fit in to the persona/archetype categories you’ve already established. You should aim to find testers who are as close to your personas as possible to ensure you’re testing the right markets.

Step 3. Set metrics

Before you can effectively run a testing programme, you need to establish which metrics you are looking to explore, in relation to your objectives. You will also have more context once you begin to establish your user test. Some UX designers and planners start by establishing metrics, others formulate them while making their user test. If you are new to creating a user test, you are likely in the latter category. If so, read on before establishing your metrics.

Step 4. Combine your methods — SUS, Multivariate, Measurable Requirements

While some UX planners choose to use singular methods during testing, it pays to combine elements from various forms of testing. These are three of your best options.

System Usability Scale (SUS)

Pose statements to users and give them the option to pick the extent to which they agree or disagree by choosing a number from 1 to 5.

For example:

Stack Exchange, DOR: 7 December 2016,

Multivariate testing

Mutlivariate testing allows you to create more than one iteration of a design and get users to identify their preferences and why. These are particularly useful for visual elements, calls to action and text length. Here’s an example, which also illustrates the subtle difference between multivariate testing and a/b testing.

Dynamic Yield, DOR: 6 December 2016.

Measurable requirements

This testing is done while the user is interacting with the site or application. You can either remotely monitor their screen and actions or manually note how often each instance occurs. These are tangible and measurable activities that can help you gage how many moments of frustration a user experiences, which parts they are able to operate effectively etc.

Some examples of statistics to note, include:

“The time it takes to complete the task;
The percentage of the task completed;
The percentage of the task completed per unit time (speed metric);
The ratio of successes to failures” (Sauro, J. 2011).

Step 5. Embrace the Findings and Implement

Once you have sufficient feedback, you need to embrace the findings and establish a feedback priority, starting with issues that came up most often or which were most prevalent. Work together within your team to find creative solutions and make the experience better for the user, in light of their specific feedback. Ensure that there is time set aside for new iterations within the project plan.

By carefully vetting participants and establishing a thorough but concise test, you’ve already won half the battle. Next time, we’ll explore implementing this user feedback within a team of multi-disciplinaries, and how to make the most of the time you spent testing.


Have questions? E-mail: studio@nonacreative.com

Was this helpful? Please click on the heart, or share your response below.

About the Author

Kerry Beetge is a Digital Strategist at Nona Creative — a full-service app design studio in Cape Town, South Africa.

Sauro, J. 2011. Measuring Usability With The System Usability Scale. Available: http://www.measuringu.com/sus.php [2016, August 10].

(Kerry Beetge-Schutte, Perfecting User Testing, http://bit.ly/2hfMUzQ, Published: 07/12/16)