15 things to do when running a User Test
So you’ve got this idea — the best idea in the world. You’re a genius for even thinking it up. Who wouldn’t buy it? Everyone wants it. It’s brilliant, amazing, spectacular…
Hold your horses there! Have you even gone out and asked your users if they want your amazing idea?
“Well no, but—”
Have you walked through every decision a consumer makes when in a situation where they could buy your amazing product?
“Well no, but—”
Have you tested to see which version of your amazing idea they prefer more?
“…”
What made you think that was a good idea in the first place?
“…I had a good feeling about it. Doesn’t that count?”
Put simply, no. Anyone can have an idea. Odds are, if you’ve thought about it, so has someone else. Ideas are meaningless without valuable data backing them up. You really think your idea is great? Figure out if it’s in demand. Go ask questions, study your target users, understand how they think.
Perform a user test.
Not sure how? Worry not! I’ve put together a guide of 15 essential things to know when running a user test (plus some bonus content)!
1) This Isn’t A Usability Test
What’s the different between a user test and a usability test, you ask?
User tests study the behaviour, interests, and emotional responses of the end user. Typical questions revolve around current user habits, how much they’d spend, when and why they use the products they do, etc. These questions help gather important data to make decisions to key points:
- Is there a demand for this idea?
- What’s the market for this idea?
If your idea gets a round of thumbs-up, then you’re onto the next step.
Usability tests focus on how the product performs within your market. These are typically executed to test certain product features, improve the UX, clean up any confusing elements, and fine-tune the experience.
Think of it this way: User tests are about the people—usability tests are about the product.
2) Keep Your Questions Relevant
The biggest part of User Testing is asking the right questions. The whole process is basically one big interview.
When creating your questions, ensure they’re relevant and meaningful. It helps to have a mission statement of what information you want to take away from these user tests—what do you want to know most? Compare your questions to this mission statement—if it doesn’t provide an answer, it’s not worth asking (at least not yet).
EXAMPLE
If you’re creating a new kind of toothbrush, your mission statement will probably pertain to dental hygiene. You may ask about flossing habits, but asking “what foods get stuck in your teeth?” might be too off-topic. Does it matter if it’s a chocolate bar or a piece of popcorn? That’s for you to judge.
*BONUS TIP*
When making your questions, order them from most important to least important, if you can. You don’t want to run out of time before getting to the point of the test.
3) Walk Through Your Test … Twice
Nothing is more embarrassing than asking a question you already have the answer for. Time is sensitive and you want to get the most out of your test.
If you have any interactive components to your user test, ensure that you have the tools ready to go. If you’re sampling reactions to a video, ensure you have working internet, and that your links are correct.
Be prepared for anything to go wrong, and have a fix for it.
Walking through the test also helps when scheduling time slots for users.
4) Test Early — Test 5 People
User testing should be one of the first things you do with your amazing idea once it’s agreed on. Otherwise you’ll put in a bunch of work developing a product people might not even want! Test early.
But 5 people? Surely that’s not enough. You want an accurate sample size!
Believe it or not, 5 people is actually the ideal sample size when deriving user insights. Take a look at this very simple yet scientifically accurate graph:
Here’s the idea. Testing 5 people typically derives 85% of the insights needed. For usability testing, that’s solving 85% of the problems, but for user testing, that means understanding 85% of what your target users are thinking.
That’s a pretty significant number. If you really want, you can test more. But there’s no financially sensible reason to do so. 5 is all you need.
5) Schedule Extra Time
When you walked through your test, did it take 40 minutes from beginning to end? Then schedule an hour. Did it take 15 minutes? Schedule half an hour.
Always schedule a generous amount of time (at least 15 minutes) on top of your user test. This will allow you to ask any extra questions that may come up during the test you hadn’t thought of, or avoid any extra hiccups in the plan.
This way, you can ensure you ask all the important questions. It’s just generally a good idea to schedule extra time.
6) Make the User Comfortable
Alright! So you’ve set up the test, you’ve got your target user in the room, and everything’s working great. Questions are ready, script is up, media is plugged in. You might want to jump right into the first questions: name? Age? Gender?
But before you launch right into the test, get comfortable. If you treat the event like a serious data collection meeting, the user will pick up on that and get nervous. This can impact the quality of their answers and they may not be able to think straight.
Take a moment to breathe, introduce yourself, crack a couple (non-controversial) jokes, and be friendly. This isn’t a job interview, it’s a conversation. A dialogue.
*BONUS TIP*
Food is a great incentive and huge tension diffuser. Want to make your user comfortable? Get some donuts, give them something for their trouble.
7) Record Everything
This is imperative to testing. Whether it’s audio or video, you should always record the words a user is saying.
Why? Live note-taking is difficult, and you don’t want to interrupt a user’s thought process so they can repeat a point for you. Instead, jot down the time stamp where you missed data, and come back to it in the recording later.
IMPORTANT!
If you’re recording someone, you absolutely 100% must have their permission to do so. A simple contract can be whipped up easily. Make sure to explain:
- What is being captured (Audio/video/photos)
- Where it’s being stored
- When it will be stored until
- What it’s being used for
The user needs to sign this before you begin recording. Happy testing!
8) 1 Tester, 1 Note-Taker, 1 Recorder
The ideal testing squad has 3 members. Someone who asks the questions and leads the discussion (Tester), someone who writes down the key takeaways from each question (Note-Taker), and someone who records the test (Recorder). Beyond just hitting “Record”, the Recorder is also responsible for taking pictures and documenting the test as it’s happening, if applicable.
Depending on what’s needed, the Recorder can be combined with one of the other roles. DO NOT merge the Tester and the Note-Taker. The Tester should be completely invested in the conversation the whole time.
9) Ask Broad & Specific Questions
Here’s the trick. Questions must be focused and precise. Often, each question is geared to extract one specific answer from the user. But you don’t want to limit the ways the user expresses themselves, so you must also be broad.
EXAMPLE
“Do you use floss?” is specific and restrictive. Don’t ask this. It limits the user to a yes/no answer, doesn’t provoke why, and may ignores other possibilities.
“Do you use any dental hygiene products other than toothpaste?” is better, it gives the user flexibility to explain which methods they use.
“Why do you use dental hygiene products other than toothpaste?” is the most optimal question—it gets into their mind, supplies the desired information, and leaves plenty of room to explain themselves. While still specific, it’s broad enough to provoke discussion.
*BONUS TIP*
“Why/How” questions are best for user testing. They force users to explain what they’re thinking—data otherwise unobtainable from the Tester.
10) Don’t Guide the User — Remain Unbiased
Leading questions are terrible. Don’t do them. A leading question is “do you think this product is good?”. A non-leading question is “How do you feel about this product?” (Notice the “How”!)
As soon as you start guiding a user to the answer you’re looking for, you’ll get the answer you expect to hear, not the honest one you need to hear. Humans like to avoid conflict, so will naturally answer in a way that agrees with your position. Remain unbiased at all times.
11) There Are No Bad/Wrong Answers
Any dialogue is good dialogue. You will never get useless information when a user explains themselves. It may not be exactly what you’re looking for, but the information is still valuable. Write it all down. You never know where it might come in handy. If the conversation is de-railing—don’t show it in your face. Remain inquisitive, fascinated.
*BONUS TIP*
If you really need that answer, try re-phrasing the question. Make it sound fresh but still relevant without making it obvious to the user.
12) Have Patience
You’re doing great. You’ll get through this. The user may keep giving answers that aren’t useful, but it’s up to you to help them stay on track.
People are often silent when thinking of answers. Let them think. If things go slow, let them go slow. Don’t rush the user—that’ll only stress them out and reduce the quality of their answers. Make them feel they have all the time in the world, even if they don’t.
13) Take Notes On What They Do
This one’s for you, Note-Takers! If you find the user is shifting their hands a lot, shaking their leg, avoiding eye contact, or behaving funny, write it down. This could indicate they’re lying about their answers, or are uncertain with how they’re responding. Physical indicators can completely change the meaning of an answer.
*BONUS TIP*
If the Note-Taker ever picks up on a weird behaviour, point it out. You’re allowed to speak too. If the Tester doesn’t catch a signal, say something yourself. “You didn’t sound too sure, would you like to elaborate more?” is a perfectly acceptable comment to make.
14) Treat Your Users Like Royalty
You’re done! Well that was a relief. Don’t run out too quickly though. Make sure to thank the user for their time, and be sure to reward them with something. This could be in the form of food or gift cards—either way, compensation is essential. Make them feel like you appreciate them.
15) Have Fun!
Okay, this one’s a bit of a cop-out, but it’s also the most important part of the process. Don’t take yourself too seriously, or you’ll spoil the fun of the test.
If you’re having a good time, so will the user. Be positive, happy, funny, and don’t stress too much if you don’t get everything you need. That’s what the other tests are for! Remember you’re having a conversation, not an interview. Be personal and open, let loose and have fun.
BEFORE YOU GO — Next Steps
Now that you’ve learned so much here, you’re ready to become a user testing pro! But sometimes the first time can still be nerve-wracking. You’ll have many questions like:
“How do I find the users?”
“Where do I interview them?”
“Should I use surveys?”
“I’m not good in conversation, who can do this for me?”
We at SignalUX specialize in the design sprint process. Part of that process is user testing, where we go off and get the data needed to make sure the idea we build is relevant, in-demand, and accurate. This is a lightning-fast process, and gets you from idea to product in just 1 week.
If your tests aren’t working out, you keep hitting dead ends, or you’re struggling to define why users should use your project, give us a shout! We’d be happy to nudge your idea in the right direction and give it greater clarity.
