How to Run a Cheap, Fast, & Incredibly Useful User Test

User testing isn’t rocket science. In fact, it’s common sense wrapped in fancy language. Unfortunately, every discipline tends to make itself more and more complicated the longer it’s around, even if the added complexity doesn’t add much.

In this short guide, we’re going to cut out all the crud and give you everything you need to know about user testing — all in less than 10 minutes. We’re going to tell you what user tests are good for and how to run them. We’ll also provide sample scripts and templates that you can copy and paste for your user testing purposes.

What are user tests not good for?

• Figuring out whether you have a good idea.
• Figuring out whether people will use your product.
• Figuring out if you have product/market fit.
• Deciding which startup or product idea you want to pursue.

What are user tests good for?

• Figuring out whether your product communicates clearly.
• Figuring out whether people can successfully use your product.
• Figuring out which features are discoverable, and which are hidden or hard to find.
• Figuring out where and why people are getting stuck in your product.

In other words, user testing is good at uncovering usability issues and bad at discovering good ideas. You should use it after you have an idea and want to make sure that you’ve designed and built it well.

Not just any design, either. User testing is useful only after you’ve come up with high-fidelity product designs (Photoshop mockups, Sketch mockups, etc.) or a usable prototype of your product.

I have high-fidelity mockups and/or a prototype. Now what?

At this phase of the product creation process, it’s useful to take your designs and show them to some potential users. You can ask them what’s going through their minds as they look at your mockup and have them explain what they think they can do on the page.

This will help you understand if the purpose of the page you’ve designed is obvious and whether the calls to action (CTAs) on your page are clear and discoverable.

Snapchat could use some user testing to improve its usability. For example, figuring out how to add a friend or contact is quite challenging because of the hidden CTA. A user has to begin typing on the search bar on the right-hand screen to be presented with users they can add from the Snapchat directory (or their address book). A Plus (+) button or “Add friends” CTA would help with discoverability.

If you show your design to five people and three of them don’t notice the “Add friend” or “Take photo” call to action, chances are you need to make it pop more. Change the design and show it to five more people. Do these people not notice it either? Change it again. User testing is great for enhancing usability and discoverability — even when you’re just at the design phase.

You may be asking: Can I use wireframes or low-fidelity mockups?

The answer to that is No! You want to show your potential users a tangible product. Low-fidelity mockups and wireframes are too abstract. People can read whatever they want into wireframes. In other words, wireframes always look good and ideas always sound amazing. Show people an actual product that they can comment on. You should be presenting them with Photoshop/Sketch mockups or prototypes.

Mockup/Prototype Testing

Here’s what you need:

• A rough draft of your marketing website’s homepage OR a rough draft of the App Store page
• The mockups or prototype you want to test for understandability and usability
• Optional: $25 gift card

Following is the format that I like for mockup and prototype testing. This is also called an “unstructured user test.” Here’s what you do:

1. Recruit a user (this is covered later in the article).
2. Sit the user down at a table and then sit next to and slightly behind them. You want to be out of their peripheral vision, if possible.
3. Explain to them that you are going to show them a product and you want them to use it and speak aloud what’s going through their mind at all times.
4. Show them your marketing or app store page. After a minute, close the marketing or app store page and open your mockup or prototype.
5. Ask them what’s going through their head: “I want you to tell me what’s going through your head as you look at this. What do you think you can do on this page?”
6. Whenever they point out what they think they can do on the page, ask them to show you how they can (or would) do it: “Can you show me how to take a picture and add a filter in the product?”
7. If you have a prototype, you can watch them succeed (or fail) at doing the proper actions. If you have a mockup, however, you should ask them to show you where they would tap or click. Then have them explain what they would expect to see and be able to do next.
8. If a user ever stalls or becomes quiet while looking at or interacting with the mockup or prototype, ask them, “What are you thinking?” or “What’s going through your mind?” to get them to talk again. These questions are nice because they don’t introduce much bias. I’ve seen a lot of people ask product-specific questions such as “Oh, are you trying to add a picture?” or things of that sort when the user becomes quiet. This instructs and biases the user. It’s a no-no. Use a simple “What’s going through your mind?”
9. Say as little as necessary throughout the testing session. Be OK with uncomfortable silences and a lack of small talk. I’ve seen many people chat away with users or jump into explanations of the product because they feel like they should be filling the silence. Don’t fall into this trap. Your job is to observe the user and figure out where they’re getting confused (and why). It’s not a marketing presentation or a sales session. During user testing, your product doesn’t have to be understood (or used) by the user.
10. If a user ever asks you a question about the product, always respond with “What do you think?” You’ll notice that users will have a lot of questions about what they’re looking at and will look to you for help. Don’t give it to them. Their struggle is information. After all, you wouldn’t normally be there when they’re attempting to use your product. If they ever ask you a question about the product, throw it back at them: “What do you think?” or “What do you think would happen?”

When you do this, you can get all sorts of responses: “Oh interesting. I’m guessing this is a feed of what my friends are purchasing. Do you pull this from Amazon?” “What do you think?” “Oh, it looks like it’s a mixture of Amazon and Pinterest purchases in a social feed. Cool.”

That’s it. You just performed your first user test.

Simple, right? That’s pretty much all there is to an unstructured user test.

However, while an unstructured user test can give you plenty of insights about how well your product communicates to users, it won’t give you a quantitative measure of how well your users can do specific actions inside of your app.

This is why you also need something called “task completion testing.”

Task Completion Testing

Task completion is when you ask the user you’re observing to perform specific actions in your product. You record whether they’re able to successfully perform the action and what problems they ran into.

Here’s how to do it:

STEP 1: Write out a list of the actions in your system.

For example, Instagram’s actions would be as follows:

Take a picture.

Add a filter to a picture.

Decrease (or increase) the lux of the picture.

Adjust the angle of a picture.

Decrease (or increase) the brightness of the picture.

Decrease (or increase) the contrast of the picture.

Increase the structure of the picture.

Decrease (or increase) the warmth of the picture.

Tint the color of the picture.

Increase the fade of the picture.

Decrease (or increase) the highlights in the photo.

Decrease (or increase) the shadows of the photo.

Add a vignette to the photo.

Tilt the photo.

Sharpen the photo.

Tag people in the photo.

Add location to a photo.

Share a photo to Facebook, Twitter, etc.

Direct message a photo to someone.

Comment on a photo.

Like/heart a photo.

Follow other users.

Search for a particular user.

Edit your profile.

Look at your activity.

STEP 2: Ask the user you’re working with to perform each (or a subset) of these actions.

STE3: Use the following designations:

• If the user can perform them quickly and with no trouble, mark it a 3.
• If the user can perform it but has some struggles, mark it a 2.
• If the user can’t perform it, mark it a 1.

STEP 4: Ask 5 different users to perform these tasks, add up the numbers, and start working on the lowest scoring behaviors first. Those are the actions that have the worst usability.

At the end of these sessions, you’ll have a list of all the actions in your product ranked by their usability. You’ll also have a bulleted list of all the problems users ran into while trying to perform these actions. Can you think of anything more useful?!

The next optional step in a user testing session is the user interview.

The User Interview

In the user interview, you talk to the testers about their lives. Specifically, you talk to them about their current behaviors. If you’re building a shopping product like Instacart or Amazon Fresh, you can talk to them about where they shop on a regular basis. Where do they go? How often do they go? What do they buy at Costco vs. Whole Foods? What’s their least favorite thing to shop for? What’s the best grocery shopping experience they ever had? What’s their favorite grocery store? And so on. The purpose of this interview is to get an understanding of the current habits of your users, so you can build a product that fits seamlessly into their lives.

Take all the information you get in the interview with a grain of salt. We humans are great storytellers. When put on the spot, we often come up with good answers that aren’t truthful. However, if you see a trend or pattern in the responses you get across your interviews, take it seriously and use it as inspiration for a creative product solution.

Midstream Recap

To recap what we’ve gone through so far:

• User testing is not rocket science
• User testing is great for uncovering usability and design issues, but bad at determining whether you have a good idea people will use.
• The most basic type of user test is an unstructured observation.
• Unstructured observation gives you a sense of how intuitive and instantly understandable your product is.
• Task completion testing is great at understanding how discoverable and usable your features are.
• Interviewing is a good tool for understanding what people are currently doing instead of using your product.
• Take your interview answers with a grain of salt. Try and focus on what people do instead of how they feel. Feelings are fleeting and sometimes illusory. Behaviors are real.

Avoiding Bias

At this point you may be wondering if there is a reason we have unstructured observation as the first thing you do, followed by task completion and interviewing. Yes. There is. We organize it this way so that we don’t introduce bias at the beginning of the user test. Unstructured observation introduces as little bias into the process as possible. After all, you’re not saying anything to the user that could change their perception of the product. When you start telling them what tasks you want them to do, however, you’re instructing―and, thus, biasing―them.

For example, during the unstructured part of the test, the user you’re working with may not even realize that it’s possible to add a vignette to a photo. However, when you tell them, “I want you to add a vignette to a photo,” you’re telling them that this is possible, thus biasing how they see your product. This is why we structure the user testing process the way we do. Task completion requires you to talk to and, therefore, instruct users more than during unstructured observation. Interviewing requires you to talk even more than task completion. It’s the most bias-packed part of the process.

We leave most of the bias toward the end so that we don’t spoil the insights we can glean from unstructured observation.

But before we hit up against the 10-minute limit I promised not to go over, we need to talk briefly about recruiting the right users for your testing.

Recruiting The Right Users

The first step of recruiting the right users is knowing who the right users are―meaning, you have to understand what your target demographic is. Are you building a product for 18-year-old college freshmen or are you building a product for successful 35-year-old professionals?

With some products, it’s easy to know who your target users are. With others, it’s much more challenging. For example, Facebook was originally a product for 18- to 22-year-old college students. However, over time the company realized that the product was well suited for a much larger set of users: anyone between the ages of 13 and 85 with friends. Thus, while Facebook might have originally tested the product with college students, they would not have been wasting their time testing 16- and 60-year-olds.

Realize, however, that there are few products that are “good for everyone.” Universally appealing applications are the exception, not the norm. Before you start recruiting users, you should have a decent understanding of who you’re building for and reach out to them for testing purposes.

After you know who you’re building for (and, thus, who you want to test your product on), you need to find and recruit these people for testing. This is the most annoying and challenging part of the user research process. It requires a lot of grunt work and a bit of creativity.

Questions to Help Recruit the Right People

#1: Where do they hang out?

Is there a local cafe, coffee shop, or store where this user group hangs out?

Where do they hang out online? Are there forums? Facebook groups? Twitter? WeChat groups?

Ideas:

• Facebook
• Facebook groups
• Old-school forums
• Quora
• Twitter
• Pinterest
• YikYak
• Craigslist
• WeChat Groups
• Yahoo Groups

Find them online or offline and message them.

#2: Do you know people who know them?

Do you have friends who know people in your target demographic? Reach out to them and get introductions.

You don’t really want to user test on your friends, since they’ll generally feel pressured into being nice to you. After all, if they’re good friends, they don’t want to hurt your feelings. Try to test on strangers or mere acquaintances. In my experience, they’re much more honest.

#3: What do they like?

Figure out how you can incentivize them to spend 30 to 60 minutes with you.

For acquaintances and introductions from friends, you can offer coffee or lunch.

For strangers, offer gift cards or cash. Anywhere from $25 to $50 is a good range for an hour of time.

Once you’ve found your ideal testers, message away! Go to forums and private message users. Reach out to people on Quora. Ask your friends on Facebook to recommend people in your target group and then reach out to them.

Now What?

Now test away and build the greatest product anyone’s ever built. You have it in you.

I believe in you.