User Testing, what it is, the benefits and how to carry it out

Photo by Helloquence on Unsplash

What is User Testing?

During the testing session, the facilitator (that’s you!) will ask the user to perform specific actions whilst describing what they are doing as they navigate through the interface. This gives you the opportunity to watch people using your software. You can then figure out places where they might be having real difficulties or where the interface is not working as you had hoped.

The findings from User Tests are really helpful. They feed into working on improvements to the interface as well as creating a better flow for the user. They are also relatively easy to carry out; all you need is a willing victim, some questions and a laptop. Doing an effective User Test though is a bit less easy though, which is why I have written this article!

*I interchangeably use “software” and “software feature” but in my mind I am generally thinking of a software feature, new or existing. You are unlikely to be testing your whole application in one User Test and would be ill-advised to do so. But more on this later!

A little about me

I thought it would be useful to put together the information I have gathered and some of the things I have learnt on the way (mainly through the tried-and-tested technique of failure!). If you have any feedback please feel free to leave me a comment or get in touch. I’d be very interested to hear about other people’s experiences about carrying out User Tests.

The Obvious Benefits of User Testing

As you might imagine this is better than the alternative. Spending weeks developing a solution without testing, only to find it has usability problems, unnecessary complexity or elements that users just don’t understand or need.

User Testing is also great for finding problems with your existing product ( because of course you want to find more problems!) Asking people to perform specific tasks with your software can be a real eye-opener. It shows clearly where existing features are causing difficulties or improvements can be made.

The Sneaky Benefits of User Testing

A lot of the time interviews will be great for background information. For instance, why someone is using your software and what they like or dislike about it. They are unlikely to remember which particular button they click, or how they add more than one item to a shopping cart for example. Often we don’t remember correctly what we do and are unable to express exactly how we want an interface to react to us.

Though Interviews are better for getting context on why the User is doing things, you will also pick-up a lot about the motivation as well. People will generally expand on what they are doing as they navigate through the software if you ask them to talk through their actions.

Selecting the Right Audience / Screening

If you are developing a consumer application or one aimed at the general public then it maybe more straightforward to find test subjects. Once people have the appropriate technical skill and are potential users for the problem your software solves they will be good candidates for testing. Even better they may already be using a competitor product or have expressed an interest in your product via website or survey.

One thing to avoid, even though it is the easy option is using people within your company for user testing. They will probably (well hopefully!) be overly familiar with your product. They will have biases and prior-knowledge that is unlikely to be the same as your actual users. Remembering the mantra ‘you are not your user’ is useful when selecting people for user testing.

Demographics in Screening

“When it comes to usability testing, we’ve consistently found that the biggest differentiator in usability metrics is not demographics differences, but whether users have prior experience or are more knowledgeable about a domain or industry.”

Jeff Sauro

A few prominent UX people (Jared Spool, Steve Krug) seem to think that demographics are not really relevant to how useable the software is and is more relevant to marketing. This would be where a specific age-group, gender or segment of the population is targeted.

This is good news as it means you don’t need to worry about finding a spread of demographics for your testing. I’m personally a bit on the fence on this point. I think that at least a bit of consideration should be given to demographics as it could conceivably affect how people use your software. I would imagine a User Test on a group of 20–30 year olds would highlight quite different usability concerns than one on 60–70 year olds.

I feel the quote above is saying that domain knowledge is the most important factor. As long as you have screened for people who are definitely potential users of your product i.e. familiar with the problem your software solves then potentially demographics can be ignored.


A clear task like this will make things easier for the user. You can also set up some Key Performance Indicators (KPIs), things that you want to track to see if your design and development work is actually helping your users.

Test whenever you can and when practical to carry out, at pre-development, at beta or on an existing product feature. Also, a sneaky technique I learned from my college course is to run a comparative test against your competitors product. If it’s starting to sound like I am an advocate of sneakiness, it’s because I am!

Prototype Preparation

With the likes of Adobe XD, Sketch and InVision you can quite easily put together some reasonable fidelity prototypes. I’ve found some effects and micro-interactions are not worth the effort currently at the prototype stage. These would be things like hover effects, browser outlines and some of the more subtle transitions. I’m personally happy to leave these for when we are actually developing the feature. It takes about as much time to add them in the prototype as design them in HTML and CSS in my experience. Though as soon as they can be done super-quick in the prototype software I’ll probably stick these in as well*.

*The lines between prototype software like Sketch and code-generating ones such as Framer X seems to be blurring a bit. I’m waiting to see which one wins, fingers crossed it’s Adobe XD as I find this easiest to use!

Carrying out the User Test

  • Try to avoid prompting, leading and closed questions
    You may know that you shouldn’t say “how brilliant do you think my button is?” Something like “would you usually scroll down at this point?” is a bit harder to avoid and also still pretty leading! Closed questions can be useful, just remember if you ask a question that is closed you are likely to get a yes/no answer.
  • Avoid providing answers
    It can be hard and sometimes you feel like a bit of a sadist when people are having real problems completing a task. Try to give them as much time as possible to figure things out themselves. If you help too early or direct them to the right area of the interface you may be a nice person but will unfortunately not learn quite as much.
  • Dissociate yourself with the design
    For honest feedback it’s generally better that people don’t know the design is your work, if it is.
  • Don’t ask users opinion too much on the design.
    Users are not designers (unless they are!). Instead focus on where they are having trouble using your interface and work on design improvements from there.

That’s a Great Question

Ease into the test with a few general questions. A bit of background about the user and their role. How they solve the problem at the moment and why they might be interested in the feature. These are all good questions to help them relax and get into the swing of things. You can also get some useful insight into their motivations at this stage.

The following are the type of questions I have found useful for the actual User Tests. They are quite straightforward. For example, what would you do next and where would you click now.

Examples of User Test Questions
What do you expect to see when you click here?
How would you achieve..
Where would you find..
What is the first thing you would..

Concluding Questions (for more opinionated feedback)
What did you like or dislike about this..
Was there anything you could not see you would have expected to..
Is there anything you think could improve this experience

How to conduct the test

For remote testing which is the bulk of what I have been carrying out the usual conferencing software such as Zoom or Skype in combination with Adobe XD or InVision prototypes has also worked effectively for me.


Careful preparation of your prototype is important but I would put more emphasis on a clear research goal and having questions that will provide as much insight into this as possible.

User Testing is a really useful research technique and I would encourage people to give it a go. I think it is surprising the amount of insight you can gain in even quite a brief session of half an hour or so. If you find the idea of jumping straight into a customer User Test to be a bit scary maybe consider a mock one with friends or family.



I’m a Product Designer based in Cork, Ireland

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store