Usability Testing: Discover What the Data Can’t Tell You
This blog post was written with Yam Goddard following a meetup for the Aleph portfolio designers community.
Although usage data and analytics go a long way in explaining digital experience usability, there are some key behaviors, contextual effects and perceptions that numbers simply don’t show. When launching a new experience, there is no usage history. While analytics explain what’s happening, usability testing explains why.
The good news is that usability testing is not as difficult as you might think. I hope this post will get you excited about user testing and what it can do for you. Not only will you become a better designer using it, but you and your teammates are likely to become addicted to its power very quickly.
We recently held a usability testing meetup at Aleph, hosting product designers from across our portfolio. After discussing the subject with such a diverse group of professionals and hearing their personal experiences, it was clear there are a number of recurring issues. The challenge mainly involves harnessing this powerful design tool, and not getting bogged down by the logistics, time and budget limitations of startup life.
During the meetup, we demonstrated user testing with audience volunteers as well as a product design from one of our portfolio companies. The conditions were not optimal. The scheduled test facilitator caught the flu and couldn’t make it, and was replaced by an arbitrary surrogate who was hastily briefed. The testing participant, a designer rather than a bizdev professional, was not representative of the product’s market fit , and the setting was extremely unnatural, with the crowd watching every move. Nonetheless, the test produced useful insights that were incorporated into the product’s final design.
The setup was super simple, and included a computer, notepad, pen, table and two chairs on the hardware side. We also utilized a basic click through Invision (prototyping software) prototype and an application that recorded both the computer’s camera and the screen, in order to capture the tester’s expressions and interactions.
During the test, we discovered that the designers of the product, a data analysis tool, were so deeply immersed in the project that they took for granted that users would understand the professional terms. The tester, while not an exact match, was a layperson, just like the target audience, and found the terms confusing and incomprehensible.
In the feedback we gathered from participants afterwards, it became clear that while some of them conduct usability testing on a regular basis, those who don’t avoid it because of concerns about complexity, effort and budget. Watching the live session helped them realize that it isn’t as difficult as they thought, and that some testing is better than no testing at all.
Usability testing is a powerful tool that helps interpret analytics and evaluate decisions during the design process. It helps resolve arguments by putting claims and theses to the test. It can also be used for analyzing competitors’ products and for supporting, promoting and selling designs to other members of your team.
We know it’s daunting, so we’ve compiled a guide to help you get started. We’ve provided quick and easy tips that allow you to conduct the testing with minimal time, budget and effort. We’ve defined practices that any company can manage and that don’t require special equipment or facilities, such as usability labs or eye-tracking hardware.
The guide also explains why usability testing is so important, and offers practical advice on common pitfalls, including when and how to conduct the tests, the scope and number of participants and how to recruit them, and how to analyze its results.
Here it is: Usability 101
You might feel overwhelmed, but you don’t have to do everything at once. Get started, and start small. I’ll leave it to Yam Goddard, a product manager at TinyTap, a social platform that empowers families and teachers to create personalized learning apps for kids, to describe how she fell in love with usability testing:
“It’s all the buzz these days. Any podcast, article or book will tell you that to create products that people love, you need to really know your people. As a social psychologist, research and human behavior is my forte. In order to implement research at TinyTap, I had first to move from an academia mindset to a lean, startup mindset. I had no choice. We were not getting the whole story from the numbers, so we needed to find a way to better way to understand our little learners.
“I decided to start somewhere, small as it may be, and progress from there. I sent out a sincere email to parents who had registered for TinyTap but had never experienced it because they hadn’t paid, asking for their help in exchange for a free annual subscription. All they had to do was film their child playing TinyTap and then send me five one-minute videos. It’s important to emphasize that it’s our obligation to keep all information connected to the children completely confidential.
“I was surprised to discover that parents were happy to help. The videos started coming in and it quickly became apparent that we were missing a lot about our learners. For example, when children ages 2–3 were asked in a TinyTap game “where is X?” they would point at the screen instead of tapping on the correct answer, and say, “here it is!” This is normal behavior. If you ask a child, “where is the tree?” they point to it, rather than touch it. But instead of engaging the child to tap, this led to an extended period of time of no interaction with the game, leading to the conclusion that the game was flawed.
What was also very noticeable with the majority of children was the difficulty they had holding the device in their hands. It was too heavy for them, and they would rest the device on their laps, on the floor, or on a table, and then use both hands to interact with the screen. This is more significant than you might imagine. Their tiny hands are still developing motor skills, and so they experience different usability frustrations than adults. For example, they would often press the screen by accident because their hands were placed on the edges of the screen, or tap an area with multiple fingers. This triggered negative feedback that discouraged them from engaging in the game.
“Another thing we discovered was the impact of the setting and context surrounding the child playing TinyTap. There were usually distractions, such as a TV on in the background or a sibling demanding attention. This made it very difficult for the children to hear the game instructions and understand what was expected of them. So we started implementing changes that would suit these circumstances.
“I could go on and on because the insights just started piling up, prompting product changes and strategic planning, all of which provided value for the product and the company. Of course, I think I’m the main beneficiary of this process. Professionally, I made better decisions based on my customers’ actual needs, behaviors and motivations, hopefully making TinyTap more child-friendly and encouraging a love for learning.”
