How we do user testing at Mirabeau
If you work in digital product design, you are probably familiar with the concept of user testing.
Putting a prototype in front of real people lets you to find out whether they actually like the thing you’re building. It also allows you to uncover all kinds of usability flaws before releasing your product or feature to (potentially) millions of people.
However, my experience is that user testing is usually the first thing that gets scrapped when money or time (or both) are running out. Which—this being digital product development — happens more often than not.
A big reason why user testing often gets axed is that (at least traditionally) it is seen as expensive and time-consuming. The process of preparation, recruiting participants, running the actual test and analysing the outcomes can easily take a week, and I’m not even talking about reporting back to stakeholders.
At Mirabeau, we see user involvement as an inherent part of our design and development process. We rarely release anything into the wild without subjecting it to at least a test or two. Ideally, we bring people in throughout a product’s development process (e.g. once every two sprints).
That said, very few of our clients have limitless budgets, so in order to make continuous user testing viable in our day-to-day projects, we have developed our own approach to user testing that is efficient, affordable and consistently produces great results.
Our user testing approach is based on a few fundamental ideas:
- We encourage our designers to run their own user tests. This eliminates the overhead of bringing in a dedicated researcher, and allows us to run users tests whenever we want.
- We prototype everything anyway. With tools like Invision and Framer as core part of our designers’ toolkit, we rarely have to put additional effort into creating “testable” prototype.
- We use a dead-simple testing setup. Using standardised, widely available hardware and software makes setting up a “test lab” a breeze for all our designers.
This article will focus on that third bit. It explains how we usually set up our test lab, and how it helps us build better digital products.
Why we like our testing setup
Our user testing setup has several advantages:
- It’s cheap, portable and easy to set-up.
It requires only free software, and no extra hardware beyond the devices we already use. This means we can run user tests wherever we want, including our client’s place.
- It allows real-time observation.
It allows others to observe the test while it’s happening. This helps to build a shared understanding of where a product needs to be improved, and often removes the need to write big reports.
- Everything is recorded
All our testing sessions are captured on video—both the participant and their interactions with the prototype. The videos are great as a future reference, and can be easily shared with anyone.
What our testing setup looks like
How it works: our interviewer (again, this is often a designer) and the participant sit together in one room, while the session is recorded and streamed to the observation room. (Thanks to Ewout for the illustration).
What’s needed: hardware and software
Our setup requires two (free) apps, two laptops and a (mobile) test device:
- Lookback (free beta)
A great Mac app that records the participant (via the Mac’s built-in webcam) and their interactions with our prototype (via screen mirroring). It merges both feeds into a video, and uploads it to the web.
- Skype (free)
We use Skype to share a screen with the observation room, where team members and stakeholders can watch the test as it happens.
- Test device (iPhone/Android, iPad, Apple TV or Mac)
This is the device that’s running your prototype. We often prefer testing with mobile devices, but that also depends on the intended audience and use cases of the product we’re working on.
- Recording Mac
This Mac is running Lookback for recording the test, and Skype for screen sharing with the observation room.
- Observation Mac
This secondary Mac only needs Skype to accept the video call. (When you have a lot of people in the observation room, you might want to use a projector or big tv screen and a good set of speakers.)
Note: if you don’t have other people to observe the test, you can obviously skip the observation room. This makes your setup even simpler, as you won’t need a second room, laptop and Skype. In any case, the recordings are always available online, so others can watch the videos whenever they want.
How to set it up
- Connect the testing device to the recording laptop (iOS devices need a USB cable, Android devices can connect wirelessly). Start Lookback, and select the testing device from the dropdown menu.
- Using Skype, set up a voice call with the observation room, and start a screen share session.
- Press “Record” in Lookback.
That’s it! It literally takes a minute to set up. Make sure that you have a fast, reliable wifi connection for the screen sharing to work, so if that’s not available you might want to resort to old-fashioned wired network.
Tips and tricks
To wrap up, here’s a few tips to help you make the most of your user test:
- Improve your prototype throughout the day. In order to get the most out of your user test, we often try to make improvements to the prototype during the day. After all, what’s the point of seeing every participant struggle with that sophisticated navigation menu? If you have a spare team member available, put him to work! In many cases, even small changes to copy or interface text can make a big difference.
- Schedule time between participants. Running a user test is intense. And as an interviewer you need to be on your toes throughout all sessions. Scheduling 15-minute breaks between sessions gives you some time to cool down, reflect and finetune your interviewing technique when necessary. It also builds some flexibility into your schedule to deal with participants that come in late.
- Wrap up with a debrief. At the end of the day (that’s 6 or 8 interviews), we usually sit together with the entire team to recap the most important findings of the test. Doing this together helps to build a shared understanding of the product’s biggest flaws. It helps to have a whiteboard nearby to sketch out ideas for improvements when needed.
- Create a summary video. This is a great way to maximize the impact of your test findings across your team. Download the source videos from Lookback, then create a 60-second montage of the day’s most important findings, using the participant’s actual feedback. There’s no better way to get a product owner or executive’s attention than seeing their beloved product or feature fail in the hands of real people.
If you’d like to get started with your own “test lab” or want to improve how you run your user tests, these articles may be useful:
- Remote User Research & Usability Methods (uxmag.com)
- How to Perform Your Own Lean Mobile Usability Testing (uxmag.com)
- Improving Your Website Usability Tests (smashingmagazine.com)
- Optimizing Your Design For Rapid Prototype Testing (smashingmagazine.com)
How do you run user tests?
While this approach works well for us, it’s obviously not the only way to run a user test. Depending on your research goals and the hardware you have available, apps like Silverback, Jaco or Verify can also be tremendously useful. And of course, you can always wander into your local Starbucks and let people try your prototype in exchange for a cappuccino.
Anyway, I’d love to hear what your testing setup looks like. What is your favorite testing routine, and how do you make sure to get the most out of your participant’s time? Leave a comment below, or hit me up on Twitter at @nickvanderlinde.
(If you enjoyed this article, please consider tapping the ❤ button below!)