User Testing on the Cheap: Excuses Kill Unicorns
Here at FreshBooks User Testing has saved us from the horrific embarrassment of delivering broken software experiences to our customers on more than one occasion. It can help catch important usability problems and be an effective tool for validation during a product design sprint (for more design sprints see our post on How FreshBooks Got It’s (Design) Mojo Back.
For a long time we didn’t actually do user testing. We had a long list of excuses for why not — it takes too much time, it costs too much, recruiting is hard, we need an expert to do it right. These excuses were so common they felt like mythology, handed down from generation to generation, you’d think user testing was some sort of mythical unicorn — widely believed to be awesome but rarely seen in person outside of design story books. One day we finally asked ourselves — could mythology be wrong? Over a year later and after countless failures and iteration, we have discovered that yes, conventional thinking is wrong. User testing can be cheap, fast and easy and you can start doing it today.
This post is designed to give you the tools and confidence you need to finally close the book on excuses, mount your unicorn and start surfing the user testing rainbow.
Recruiting is Time Consuming
Reaching out into the world to ask for help from complete strangers can seem like an impossible task. The reality is people are very generous with their time when they know they’ll be helping others. To find people to test with we do things like emailing out customers or posting to Craigslist to find people who have never heard of us. To get people to respond to our emails we continuously test and improve our email template using Mailchimp to track open rates and test different subject lines. Even if people confirm their participation in user testing, you should expect at least one out of every four users to cancel last minute or just not show up on testing day. If you want to test with four people, (a widely advocated sweet spot) make sure you confirm five or six testers but remember not to book any of these testers back to back, unless you’d like to try running 6 tests simultaneously.
However far in advance you do recruiting it still takes time. To help alleviate administrative pain we’ve automated as much of the workflow as possible, one specific tool we’ve found helps with the painful task of coordinating available appointment times and followup is You Can Book Me. While developing a recruiting process make sure you document the painful stuff, we’ve found that a quick Google search uncovers a myriad of time saving tools designed to address nearly every pain we can think of. For example, do you hate the idea of emailing people or don’t have a big user base? Ethn.io will automate the task of recruiting with a little pop up right on your website.
It’s Too Expensive
We used to think that to do user testing properly you needed an expensive facility with fancy equipment and software. In reality at FreshBooks we currently run user testing in a spare meeting room on our oldest Macbook. We use cheap off the shelf screen capture software instead of a two way mirror — Silverback to record local tests and Screen Hero to record tests outside the office. We compliment our testing with services like Usertesting.com where you can recruit and run unmoderated tests that come back as videos in as little as fifteen minutes for about $50 a pop.
The largest expense and by far the most important is making testers feel special, we give everyone a gift card to thank them profusely for their time. This also helps motivate them to respond in the first place. To ensure the quality of our Craigslist testers, yes unsurprisingly people surf Craigslist looking for quick cash, we use a simple screener we host on Survey Monkey.
It Takes Too Much Time
We’ve all been in a situation where we overcommit or have been forced to design on a short timeline. In this scenario you get to the end of the project before realizing you forgot about user testing. Typically there’s no time to recruit testers, no time to develop a prototype and run a test and even less time to synthesize feedback and iterate. When you alert your team they sigh and tell you “there’s just no time, plus everyone’s happy with the design! Relax…”
At FreshBooks we solved this time management problem by simply making user testing a standard part of every week (as advocated in Jeff Gothelf’s Lean UX). Every single Thursday there are as many as twelve customers in the office for user testing. These testers are booked weeks in advance before we even know what we might be testing. This single change radically shifted mindsets from, should we test this? to what are we testing this week? The effect has been transformative, designers are now far less defensive about their work and we rarely argue over who’s opinion is more correct, instead we use insights and validation from real customers to moderate vibrant discussions about the success or failure of our weekly hypotheses — the best use of our time!
As for prototypes, we use InVision most of the time, a freemium web app that makes building clickable prototypes dead simple. Drag, drop, done. If InVision’s toolset is too rigid we build paper prototypes, or if we have a little extra time build something dirty in code – whatever it takes to have a conversation with the user and observe their interaction with the product.
We Need an Expert
User Experience Research is a relatively new field, and one that emerged from academia, making it appear to most that if you didn’t study it in school you couldn’t possibly do it yourself. The truth is that just a few tips and some real world experience will get you well on your way to being an expert. Any expert will tell you that user research is best experienced first hand rather than in a dusty report or third hand debrief. To get us over the knowledge gap quickly at FreshBooks we read Steve Krug’s Rocket Surgery Made Easy, it’s very short and full of absurdly practical advice, including actual testing scripts and setup advice you can use to get started right away.
Don’t have time to read even a short book? Then please note the most common amateur pitfall — Designing tests that lead a user or produce false-positive results. To make your tests less prescriptive try giving a user a high level scenario and task rather than a list of steps to follow.
Just do it!
This post is short for a reason, we know you don’t have a lot of time but user testing is too important for you not to be doing it. Hopefully we’ve given you the confidence to get started. There’s plenty more to know, and mountains we’re still learning too, but the most important thing right now is that you stop making excuses and start doing it.
Jeremy Bailey is the Creative Director @ FreshBooks.com
Sound intriguing? FreshBooks is hiring UX Designers, apply today and mention you read this post for bonus points!
