Ask Women in Product: How do you do usability testing?
Amelia Abreu and Amanda Tersigni share practical and actionable advice for getting started with Usability Testing.
Answer from Amelia Abreu, Founder and Director, UX Night School
Usability Testing (or U-Testing) gives product teams a way to connect with end-users and build direction for future product development. There is a learning curve, but it gets easier and less time-consuming every time you do it.
If you’ve never done U-Testing before or need to sell your team on the benefits, keep in mind that improvements to usability have a demonstrated (and mind-blowing!) ROI of 10 to 1. What’s more, U-Testing — together with other forms of hands-on design research such as user interviews — is a powerful tool for consensus building, and aligns product teams to better serve their users.
While each PM or design researcher does things differently, there are five main steps in the U-Test process. I describe each step and link to some of my favorite resources below.
1. Plan and Scope the Project
On average, it takes about 40 hours (spread over two to three weeks) to test your product the first time. And, let me restate this: every time you do it, it gets easier and less time-consuming.
The most important step in the planning process is deciding what to test and defining what these tests will be like. Maybe you have new features you want to test, or you might have an existing product you want to improve. In all cases, successful U-Testing is focused, i.e., we ask our participants to execute two to three defined tasks.
When I work with client teams, I often use this one-page Usability Test Plan dashboard to map out where we want to go and how we plan to get there. Another great resource is this Research Iteration Template, a Trello board created by Jessica Crabb, which does a great job of breaking down each step into manageable (and trackable) tasks.
2. Recruit Your Participants
How many participants do you need? Meta-studies show that you can uncover 80% of usability issues with as few as five test participants. Recruit five folks who are demographically representative, i.e., they reflect the diversity of your core users. Use UX Night School’s recruitment cheatsheet to learn how.
How long does recruitment take? As a general rule, I allow two weeks before a test to recruit participants. While you can outsource this task to a professional recruiter (I’ve worked with Forage Research and recommend them), I’ve found that taking on participant recruitment myself is an insightful part of the process. At the very least, you’ll get to test your hypotheses about who your users are, where they are, and why they exist.
Participants are a valuable resource, so treat them well and let them know that they’re not being tested, but rather, the product. It’s a good practice to pay your testers for their participation and to establish informed consent.
3. Align Your Team
Involve other members of the product team in the Usability Testing process. At a minimum, you’ll gain multiple perspectives on your observations, thereby creating a richer understanding of the problems you encounter. Moreover, the act of getting together to observe and participate in test sessions helps your team build consensus and productive working relationships. Have the PM or design researcher serve as the moderator while the rest of the team observes. It’s a humbling and revelatory experience to simply watch and listen.
To paraphrase The Lean Startup, get out of the building! You don’t need a dedicated “usability lab” — I find the two-way mirror setup creepy! Instead, try to observe participants in their everyday environments. You’ll discover insights you would not get in a controlled lab environment.
It’s a good idea to record sessions, but don’t rely on the recordings as a substitute for participation. In my experience, the team members who say they’ll “watch it later” rarely do. If the team can’t participate in sessions, organize a sync-up meeting to share recordings and recap the learnings from these sessions — that way, everyone is on the same page.
4. Track your Insights
It’s easy for things to become a blur when you conduct several test sessions in a day. Schedule enough time to debrief with the team between sessions, and use this time to capture insights. My friend, Susan Farrell, shares techniques for collaborative notetaking during test sessions; I’ve also used Tomer Sharon’s Rainbow Spreadsheet with great success.
While it’s valuable to take time after a round of testing to dig deep and analyze your findings, you can get a step ahead and build consensus by tracking observations as they happen.
5. Iterate: Plan the Next Round
Now that you’ve carried out your first round of U-Testing, use the momentum to plan for the future. After all, the most resource-effective approach to U-Testing is to make it a regular part of the development cycle. For example, I worked with a team that dedicated every other Thursday morning to U-Testing, which meant that each major feature development in the last sprint was logged for testing. By taking the guesswork out of scheduling and making U-Tests a normal occurrence, the team was able to iterate and ingest findings and ideas without having to drop everything.
Iterative testing also lets you track progress on key metrics over time. If you test five users per month, you’ll have 60 data points in a year and will have built a rich and more statistically significant data set over time.
The more frequently teams invest in design research, the less mysterious it becomes. With strategic planning, you will build not only a practice of usability testing, but a truly human-centered product.
Answer from Amanda Tersigni, Product Manager, UpHabit
You’ve sold your team on the ROI of Usability Testing, and you are excited to get started! Now it’s time to pick a testing method that will work for your team and product. Below I describe a few useful approaches and provide tips to get started.
Hallway Testing
A lightweight approach is ideal for testing design concepts at early stages. Hallway tests let you run random (qualified) participants through designs to gather data for your test objectives. These tests can be performed with colleagues or friends.
Here are a few tips to get started:
- Set objectives for the test.
- Prepare a focused test script.
- Meet with the participant, observe their behaviours, and log any frustrations. Note: If you are conducting this session remotely, use conferencing tools that let you see both the participant and their screen.
- Ask questions and ask the participant to reflect on their experiences.
- Take notes; be sure to remain engaged.
Hallway Testing is a cost-effective approach to usability testing which helps to validate concepts and gather insights in early stages.
Moderated Testing
A moderated testing exercise is useful in early-stage product development. The participant is recruited based on the specific requirements of the test and the target persona. A moderator is present to conduct the test according to the test script. Additionally, the moderator analyzes the progress of the test and, when necessary, takes the test off-script by driving the discussion with participants into meaningful topics.
Here are a few tips to get started:
- Set objectives for the test.
- Identify participant requirements and demographics based on the needs.
- Recruit test participants. Note: Schedule enough time for the test based on the script. Ensure that incentives for the participants are clear (e.g., offer a gift certificate).
- Prepare a focused discussion guide for the moderator and the test participants.
- Find a suitable moderator, someone who is personable and familiar with the product. The moderator should also be comfortable with probing participant behaviours and responses to gain additional insight when necessary.
While moderated tests cost more and can be time-consuming, they typically uncover unexpected behaviours and requirements via unscripted dialog and yield learnings that are valuable to your team.
Unmoderated Testing
Use unmoderated testing exercises to collect unbiased feedback from users while they interact with the product on their own. The participant will flow through a set of goals (predefined by you) while interacting with the product. Depending on the tools used, test results come in many forms, such as screen recordings, voice recordings, etc. Since there is no moderator, the participants interact and express their feedback in a very natural way, which makes this a powerful technique to understand a participant’s experience.
Here are a few tips to get started:
- Set objectives for the test.
- Select a tool that will help you conduct this test (e.g., UserTesting or lookback).
- Create the test with clear and specific goals. Note: Since this test is unmoderated, you’ll want to provide enough detail on the steps so your participants can complete the test successfully.
- Schedule enough time to review the results, such as screen and voice recordings, to gather insights.
The unmoderated testing method is ideal for collecting feedback on products already in-market because these tests help you identify gaps and frustrations in the user’s experience.
Regardless of the Test Method . . .
Results from Usability Testing will help your team gather useful insights on products and concepts. I encourage experimentation with different methods of U-Testing depending on your budget and the stage of your product. There is no one-size-fits-all solution, so be clear about your objectives before selecting a solution for your team and product.
After repeating a number of these tests with different participants (four to five participants per session are typically enough), you’ll see patterns quickly emerge. Through these tests, the PM and design team gain a better understanding of their users and receive insightful feedback that will inform the product roadmap.
Thank you to @mdy for editing this piece.
