Usability testing on zero budget
How to run usability labs on zero budget
If someone has to stop and think in order to use your product, there had better be a damned good reason. Good product design should be invisible — so easy to use as to be almost automatic, a process as natural as breathing. This is the essence of usability.
Getting to that point is hard because design is hard. Anticipating people’s behaviour is hard. Managing resources is hard. Thankfully there is an invaluable tool that can help you on your way — usability testing.
What is usability testing?
Simply put, it’s when you get a customer into a room for an hour, sit them in front of your product and ask them to use it. You watch them muddle through and then comfort them when they’re reduced to tears of frustration, all the while noting the stuff you need to fix. If you do it right, you will learn as much in a couple of hours as you’ll learn from weeks of aggregated quantitative data.
The most common objection to usability testing is that it’s expensive. It’s true that some agencies charge thousands of pounds for their work, and a couple of them can even justify it. This guide, however, is for the world’s start-ups, bootstraps and one-man bands. Welcome to the world of zero budget usability testing.
Step 1: What do you want to learn?
Quantitative data, the raw numbers behind your product, are a good place to start looking for questions. Numbers are great, they tell you a lot about what’s happening to your product. You might know, for instance, how many people start registering for your app but never complete the process.
What numbers usually won’t tell you is why people aren’t using your product. Sometimes it’s for reasons you can’t do much about, but more often than not it’s because you haven’t built the right experience for them. Usability testing helps us see what we’re doing wrong and how we might do it better.
With this in mind, choose an area that’s going to have an impact on your product. Maybe it’s registration, maybe it’s paying for an item, maybe it’s the ability to edit a photo. Whatever it is, try to focus on one area that a hunch, or preferably some solid data, tells you is important to your users.
Step 2: Recruitment
There are a few broad types of user to recruit from: established users of your product, lapsed/irregular users and complete strangers. Each group will give you unique insight into how your product works.
Testing on complete strangers allows you to see your product through the eyes of a first time user, which is crucial when aiming for growth. These people will show you a lot more about the usability of your registration process, for instance, than users who have already been through it.
If you have a database of users, and you’re entitled to contact them by email, you can start to target at a more granular level, such as platform, device and geographical location (no point asking a user in Sydney to attend your session in London).
Once you’ve established the profile of the users you want, prepare a simple, friendly recruitment email and send to roughly 100 people per desired attendee. If you have a little cash to spend then I strongly advise including an incentive as it tends to make for a better test. That said, if you really are on zero budget, you’ll be surprised how many people will attend purely out of curiosity. Here is an example from a session I ran recently:
For recruitment of strangers, try classified ads on Gumtree/Craigslist. The quality of the applicants will be lower, so try to do some screening first. A Google form with some brief but selective questions is a good way to separate time wasters from the genuinely curious/useful.
I normally aim to speak to five people in one day, allowing a full hour for each. Make sure that your guests know exactly where they are going on the day and that they have your mobile phone number.
Step 3: The toolkit
The exciting bit! Whether you’re testing a desktop product or a mobile app, you’re going to need a way to record the session. Enter Lookback, definitely the coolest piece of software I’ve seen in a while.
Lookback turns your laptop, iPhone or Android device into a mobile usability lab. It will record your participant’s face, voice and their interactions with the screen, including taps, swipes and mouseclicks. The Android version even allows you to test remotely, so maybe that user in Sydney can get in on the action after all. Here’s a (slightly dated) demo of how Lookback works:
Best of all, Lookback is (at the time of writing) free of charge!
Other things you will need:
- A consent form — including acknowledgement of receipt of incentive, if applicable. Make sure all users read and sign this when they arrive.
- Light refreshments.
- A quiet room!
Step 4: The test script
To run the test, you’re going to need a test script. This isn’t as scary as it sounds, it’s simply a list of tasks that a user would need to complete in order to use your product. Try to avoid leading questions, keep it as broad as possible. For instance:
Don't say: “Open the app, go to the settings screen via the menu and change your email alert preferences from daily to weekly.”
Do say: “Imagine you’re unhappy with the number of email alerts you’re getting from Azimo. Please try to reduce their frequency.”
Try to put a real world use case in front of the user rather than a list of instructions. The aim is to see how they tackle the task without guidance — a real user won’t have instructions. You’ll learn much more from where they fail to complete tasks than where they succeed.
Another option is to pose a “treasure hunt” question. A good test for a clothes shopping app, for instance, is to ask a user to find a pair of brown leather shoes in their size. The way they go about completing the task will tell you a lot about how effectively you’ve designed your site navigation, search process, filters etc.
Make sure that your script fits the time allotted. Remember that you’re only likely to get 30–40 mins of actual testing time out of the hour. I usually do a dry run through the script with a couple of colleagues the day before.
Step 5: The test!
When the first user arrives, I guarantee you’ll be a little nervous. That’s normal, and they will be too! Remember to greet them warmly, offer them a drink and a snack. It’s best that they feel comfortable because you want them to act naturally when they’re completing the tasks.
Once you’ve introduced yourself and asked them a couple of warm up questions, explain how the session works:
- Explain that you’re going to get them to complete a series of tasks.
- Remind them that if they get stuck, it’s not their fault. You’ll be surprised how many people get flustered when they can’t do something, then immediately blame themselves rather than the app.
- Explain that you can’t help them in the test, unless they get totally stuck. Point out that while you won’t be able to answer their questions during the test, they will have a chance to ask whatever they want at the end.
- Ask the user to think aloud. Explain that if they talk through what they’re doing, it makes it easier for your to understand their thinking when you’re reviewing the footage. Reassure them that no statement sounds too silly.
- Ideally don’t have the designers or developers of the product in the room with you. They’ll be able to see it all on Lookback anyway!
When they’re ready, start the recording and crack on with the test. Remember that you’re looking at their behaviour for clues as to how well your product is working. How instinctively do they do certain actions? How long do they take to complete a task? Are different users repeating the same behaviour? Is there more than one way to complete a given task? Which one does the user instinctively choose? Constantly ask yourself: “how hard are we making these people think?”
Remember, the golden rule of usability testing is: KEEP YOUR MOUTH SHUT. It is so, so hard to follow this rule but it’s the most important one of all. Don’t feel the need to fill silence, let the user do that! Even when you’re desperate to scream at them. Sometimes the things you and your tech-savvy team think are super obvious, really aren’t obvious at all. For more guidance on how to talk to users in tests, see this awesome article from Kara Pernice (she also does great usability training for NN Group).
Step 6: The review
While you can make important notes during the session, it’s usually best to have an observer do that for you, so that you can focus on the user. After the sessions are complete, you can review each one and note the places where users got stuck, or where an experience worked particularly well. You will be amazed at how much you can learn from just five people. Time and again I’ve found all five getting stuck in the same places, or making exactly the same noises of surprise or annoyance at exactly the same time in the test.
Collate all your findings and I guarantee you will get at least three or four significant changes to your product that you can make almost immediately. You will find stuff that you had no idea was broken. Share the findings with the rest of the team, show them some video clips and invite them to comment. Lookback has a handy feature that enable comments and group discussions about specific points in the videos.
Finally, get to work. Take your key finding and, if you’re confident it’s validated, get it into a sprint as soon as you can. The whole team will feel the benefit of the test and you’ll ship immediate value to users off the back of it.
In conclusion
Usability testing is a cornerstone of digital product development. At Azimo we run sessions once per month, each costing just £250 in incentive money. Each month we learn something new about our users and about our own products.
Above all, sitting down with real customers reminds you who you’re building this stuff for, and why it’s important.
If you’d like to work in a creative, energetic team at the cutting edge of financial technology, check out https://azimo.workable.com — we’re hiring!
