Usability Testing Questions: Tips & Examples

Regardless of the product you design–a website, app, game, multifunctional platform, ecommerce store, or interactive tool–one of the essential truths you need to remember is that you are not your user. Even if you do belong to the target audience of the product, one person cannot represent the variety of interaction scenarios that may arise. User testing is the best way to make sure you account for that variety; it’s the stage of the creative process focused on what users really need, feel, and think, and it plays a critical part in product design and improvement.

At the core of user testing is determining what you should ask and how to ask those questions in a way that drives actionable insights. In this article, we’ll explore some guidelines for asking the right user testing questions (with examples along the way).

Basic do’s and don’ts of user testing

How do you know the right moment for the right kind of question? How should you know when to interrupt and guide the conversation versus when to just ‘shut up’ and observe? The best course of action depends, of course, on the context of each individual session. But there are fundamental guidelines you can apply to just about any situation. So, let’s start with some do’s and don’ts when thinking about usability testing questions.

Do’s for usability testing questions: Uncover user flows and feelings

DO ask users to share what they are feeling and thinking of while interacting with a product. Don’t focus only on the results of a particular task. Asking users to share their comments will also help you gain insight into the goals of the user and their state of mind; this may even surface other important insights that dramatically improve usability.

Example question: What are you feeling/thinking about when you complete this [action or task]?

DO give the destination and ask about the route. This kind of task analysis will help you uncover the mental models and interaction patterns of your participants. Strive to find out as much as possible about your users’ mental models as possible (how they complete a task). These mental models will help you bridge the existing experience of your target audience with any shortcomings (what’s hard or not enjoyable for them), and therefore you can decide on the next iterations of your product and its features. You may create a super slick, elegant, minimalist interface that looks harmonic but if users can’t find the menu, can’t search, or have no clear CTA prompts, your product will not be truly successful.

Example question: Assuming you wanted to change the e-mail address associated with your account, how would you go about doing that?

DO care about the facial expressions and body language of your users. We, humans, are emotionally-driven creatures, so when testing a product, you should also observe and analyze the non-verbal cues of participants. A word of caution: correlation is not causation. People may demonstrate negative non-verbal signs not because of the product itself, but because of their general state of mind or if they’re simply uncomfortable in their current situation/environment.

Don’ts for usability testing questions: A little less conversation, a little more listening

Don’t provide too much guidance. Your goal is to make the testing session informative and objective, so step back and avoid guiding users on how to respond.

Don’t prevent failures or missteps. If you observe a user carrying out a task in a way that is not likely to result in the desired outcome, don’t interrupt, let them fail! For example, if a user-initiated search is coming up empty, don’t say “Maybe you should try the filters for your search?”. Our goal here is to better understand how and why the user fails, so we can improve.

Don’t help the user. On a similar note, you may face a situation where the user will explicitly ask you what to do in a situation, but your goal is to encourage them to find the solution by themselves (or declare it impossible, which sometimes happens). Don’t help them find a solution as the results won’t be valuable in this case.

Don’t distract the user. Another mistake that’s easy to make is to interfere with the user’s process when they’re already deep into an interaction. If you distract them, you’ll miss the natural user flow.

Don’t talk too much. Even polite and friendly remarks or encouraging words may harm the research process. Everything you say to the participant should correspond to the goal of the testing. Don’t overload the participants with conversation; instead, let them direct their own minds and focus on the task. You are observing how they interact with an experience, not interviewing them.

Don’t ask yes/no questions. You don’t have unlimited time with participants. So, it’s much handier to use your time with them to ask questions that give you lots of information, rather than basic and uninformative yes or no questions. Ask open-ended questions that stimulate the users and get them to describe exactly what they’re doing, feeling, or thinking.

Don’t believe everything your users say. Always leave room for a margin of error. Remember: no matter how friendly, natural, and positive the testing atmosphere is, it is still not a natural environment for people and that may influence their behavior and affect their responses.

Useful questioning techniques for usability testing

There are three handy techniques for making usability testing questions productive. The approaches below come from Nielsen Norman Group, and in practice, they are handy not only for testing products but also for discussing prototypes with non-designer stakeholders or clients. Let’s briefly introduce them.

Boomerang

This approach involves smooth and non-aggressive questioning, returning the users’ own questions back to them. It is helpful when a user asks for your guidance and you need to find a way to encourage them to find the solution themselves. For example, a user that asks: “Should I sign-in to make a purchase?” You don’t want to give them direct yes or no (as explained above), so you can answer back to them with an informative question like: “What do you think?” or “What would you do in this situation if you were alone now?”

Remember, we don’t want to help the user. The above approach reinforces this guideline and encourages the user to think through the problem. It better informs how we design the interaction.

Echo

This technique is clear from its name: it involves repeating a part of a user’s answer with an interrogative intonation to encourage the user to provide more detil on their feelings or thoughts. Be careful to use exactly the same phrase the user used in their answer, they will feel the natural urge to explain in their own words.

For example, if a user says, “This button looks weird,” the facilitator can respond using the same phrase with a questioning tone: “Looks weird?” This response will prompt the user to provide more detail and clarification.

Columbo

Named after the famous film character Lieutenant Columbo, this approach helps create the perception that the facilitators of the usability testing are less aware, knowledgeable, and attentive than they actually are. Its guiding principle is “Be smart, but don’t act like it.” This way, you avoid situations where you end up teaching or guiding the user. What’s more, seeing your hesitation, some participants may naturally feel like they want to help you. Yes, it’s not an easy art, but it’s definitely worth trying.

For example, when a participant asks if the Undo button means they will lose their progress, don’t answer yes or no. Try the Columbo technique, instead, with an incomplete question like: “So, you want to know… (pause) if you can (pause)…” The chances are high that the user will then provide more specific information on why they’re unsure about the functionality of the button and the consequences of hitting it.

Make no mistake, the techniques mentioned above don’t mean you should interfere in the usability testing process all the time. Always consider the following points:

• Not all the questions users ask really need answers. Sometimes these are just rhetorical questions, or they are thinking out loud as a part of the exploration process.
• When you have enough information to make a conclusion from the user’s comments or just from observing their actions, you may not need to prompt any further discussion.

Questions before and after usability testing

Although the goals, products, and participants behind a usability test may be very different, there are questions that are helpful in any case. The process of collecting information starts before the user even begins interacting with a product and it doesn’t finish immediately after all tasks are complete.

Pre-testing usability questions

Before the participants start actually interacting with your product, you will want to learn a bit more about them so you can later define which personal factors might have influenced their experience. Here are a few useful screening questions:

• Demographic details questions (asking for someone’s age, gender, education, occupation, etc.)
• Tech-literacy questions (how often they use technologies and devices, what kind they use, how easy or hard using these devices is for them, etc.)
• How they currently solve any problems that your product also seeks to solve.
• Questions about their income level (essential for paid functions in apps, ecommerce, premium plans, etc.)

Also, the popular 5-second test technique is an effective way to gather user’s first impressions before the deeper testing starts.

Post-testing usability questions

Unfortunately, when the testing session itself is over, it’s not the time to crack the champagne just yet. The end of a session is just the right time to uncover a user’s overall impression and feelings about the product they used. Don’t ask about specific features or points; ask about the experience in general, like:

• What did you like about the product?
• What didn’t you like about the product?
• Is there any product similar to this one?
• What were your feelings about the instructions?
• Do you know anyone that would like this product? (Ask them to describe those people. For many of us, it’s easier to talk about people than about products)

User testing can provide valuable insights into how real users see your product; it allows you to answer key questions, like whether your information architecture has been crafted effectively, whether elements in your interface actually hurt usability, whether your navigation is intuitive enough and the UX copy is written clearly, etc. The best way to ensure that your testing sessions are successful lies in doing your homework. Prepare your user testing questions and you should find that you’ll be able to create a natural and comfortable environment where your users open up and help you create better products.

Visit XD Ideas for more unique insights and authentic points of view on the practice, business and impact of design.

Join a free Daily Creative Challenge to sharpen you UI/UX design skills

Learn about Adobe XD, the powerful platform where teams collaborate to create designs for websites, mobile apps, voice interfaces and more.

Originally published at https://xd.adobe.com.