Creating Effective User Testing Questions
If you have great questions to ask your users, you will get great answers to apply to your testing goal. If you work for a business that has limited resources, getting the best feedback is important the first round testing, thus saving money.
Understand the Goal
Every user test needs to have at least one goal to be effective. If you work for a company, ask the product owner, product manager, or leader what the goal for the test is. If you work by yourself, come up with a goal that will allow you to compare and measure the results.
Once you have the goal(s), understand what that goal means. For example, if your goal is to determine if there are any usability issues on an interface, you can use quantitative data to ask the user if a particular task was easy or difficult.
Think Like a User
The best advice for writing great questions is to put yourself in your user’s shoes. Pretend you are a user that takes things literally and have never seen your product nor have any idea how it works. Ask yourself these questions:
- How did I get to this product?
- What is this product for?
- What am I supposed to do?
- What will I get?
Knowing the answers to the questions will help you write an introduction for your user to get in the mindset of your test.
You were browsing your social media feed and see a title that looks interesting, so you click on it and come to our website.
Use a Prototype or Mockup as Reference
Always have access to a prototype or mockup when writing the questions for the user test. Having a visual of the interface in front of you will allow for you to see the path you want the user to take, and this will easily allow you to write the questions and the actual tasks you want them to perform.
Readily having the mockup or prototype available will also prevent you from accidentally skipping over an important step or feature of the interface when writing the questions.
Write Questions with Clarity
Here are some great quick tips to help you write clear questions.
Use Layman’s Terms
Please don’t use industry jargon when writing the questions! Your users will not understand the question clearly and will not provide good feedback.
No: What are your thoughts on the interstitial on the application?
Yes: What are your thoughts on the pop-up with the discount information?
One Question at a Time
Keep the question focused to one thing. Do not pack in several things into a single question as this gets too difficult for the user.
No: What is your opinion on how the search works and the way the results are given to you?
Yes: What is your opinion on how the search works?
Write questions in a way so you won’t get biased feedback from your users. Write your question as neutral as possible.
Wrong: Was it hard to find the search box on the site?
Right: How easy or difficult was it to find the search bar on the site?
Get the Responses You Need
When writing your questions, you have two types of feedback based on the types of questions you write:
1.) Quantitative (can be easily measured)
2.) Qualitative (influenced by feelings or emotions)
To achieve the user testing goal ultimately, it is ideal to have a healthy mix of both quantitative and qualitative feedback from your users.
Use the following types of systems to write your questions for easy to measure results from your users.
- Rating Scale (levels of difficulty, clarity)
- Yes/No (completion of task)
- Multiple Choice (to prevent a wide range of answers)
Qualitative questions are best if you ask for the user to talk out loud what they are thinking or to write their answer and talk out loud as they do. Your best feedback will come because the user gets to speak their mind and express areas on your prototype that may be confusing or cause friction.
- Find a class that is convenient for you. Talk out loud as you demonstrate this.
- Is there any other way you would find a class on this page?
- Was there anything that seemed confusing or difficult while you were searching for a class? Please explain out loud as you type your answer.
- What frustrated you most about this site?
Avoid User Test Frustration and Issues
If you have ever conducted a user test and when going over the results find that there was confusion on the instructions or questions, there is nothing more frustrating than seeing your user wander away from the page or area you want feedback on.
Here are some helpful tips to keep your users on track and get the results you need.
This is Just a Test
Sometimes you have to let your users know that your prototype or mockup isn’t functional yet. Doing so will keep them from asking questions about links that don’t work or getting frustrated when they can’t move further along on your interface.
What you will be testing is a prototype, an idea where not everything is 100% functional or clickable, so as you explore you may find things that don’t work yet.
Unexpected Results from Free Reign Testers
If you give your users free reign to click around on your prototype (if it is functional), there is the possibility that your user may move on to the next question while on the page you did not intend to test. It is important to communicate with your user to return to the specific page you want them to be when moving on to the next question by simply adding a statement (and link if you really want to be nice) to the beginning of your next question.
Be sure you are on the home page before continuing to the next question.
Allow the User to Vent
Sometimes, you can really get awesome feedback from allowing your user to vent, or tell you what frustrated them the most on your interface. The best way to do this is to allow them to type in their answer rather than talk out loud. The user will be more intentional about what they write and provide insightful feedback.
What frustrated you most about this site (remember it’s a prototype, so items may not be clickable)?
To get the best possible results from user testing, clear, concise, easy to understand questions are essential to get the feedback you want to achieve your goal. Being able to think like your user is a huge plus when writing effective questions.
Carefully crafted qualitative and quantitative user test questions are the framework for clear feedback responses that can help you see if you achieved your user testing goal or not.
Most importantly, being able to avoid user frustration while taking the test will result in honest, helpful, and effective feedback that you can bring back to your business unit, stakeholders, leaders, or product owners.