The ultimate guide to get started with usability testing for UX in 2024
Wanting to conduct usability testing is almost like looking to hire a new colleague. You put yourself out there with your offer, new faces come in with something thrilling to say, some may be interested, some may reject your offer. It’s tough, it’s vulnerable, everyone does it differently. But you wouldn’t hire someone just based on their CV, you want to get to know them first.
Achieving great product successes, is nothing less similar, and time and again, testing with users has proven to be a highly effective way to reach that. Something that at Bolden, we’re fully convinced with, and we’ve put together a guide to help demystify it for you too.
The 4 best types of usability testing
We’d like to make known that you do not need a big budget and a tech-filled lab to sustain a healthy outside-in thinking design process. Starting with usability testing depends on your available resources such as your organisation size, their design maturity, the timeline in which you’d need to acquire insights and your overall project roadmap. These 4 factors are the key difference-makers for getting started.
Qualitative and quantitative
Qualitative usability testing is a methodology first made public to a broader audience by Steve Krug in the all-time UX classic ‘Don’t Make me Think’ and its sequel ‘Rocket Surgery made easy’ — which instructs the host to invite 5 people over for an in-person testing session including interviews, viewing how these people experience the interface. But why 5 people? As NN/g describes: with about 3 you capture 90% of your usability barriers and with 2 additional people you’ll account for most edge cases.
At the opposite end sits quantitative testing, where copious amounts of results are welcomed to create a benchmark. More is more. This could be asking your extended family some questions, asking your 1000 social media followers to vote or pay 30.000 users in store-credit. With qualitative testing you’ll rally your community to improve your product.
Both have equal merit but if you are just getting started and have a digital product available; go for quality! These interviews will provide you with both quotes and testimonials.
With about 3 you capture 90% of your usability barriers and with 2 additional people you’ll account for most edge cases.
Remote or Local
Nowadays, digital connection enables us to reach millions of users from within their denim pocket, living room and virtual head-set. In asking for help on a remote scale there’s a variety of measures to undertake. You can start with buying ad space, posting on your socials, inviting users all over unto your landing page or asking a remote company to conduct communications. A service like Usersense.se can help with such recruitment.
If your product is being used in a specific region (think: public transport or governmental) it’s best to directly tap into that community. Especially if you don’t relate to your user base. With local testing you can utilise guerrilla tactics. It seems out of the ordinary in the design process but it’s a fun practise. The classic method written by Nick Babich shows the fun, low-fidelity side of testing.
With our UX team based in the center of Amsterdam, we have the luxury of stepping out and finding people from all over the world in one dense location. We can freely test out digital products, walk around embodying the user and we can interview people on the spot. This makes for a great ecommerce test, as this case study shows.
Interview or Recording
Interviewing your participant within close proximity, either present at the desk or just one step away, also goes by the terms moderated- or facilitated usability testing. The participant is guided via a task sheet. The stage is set by a moderator.
Being present during interviews allows for more context, you yourself get to experience what the user is thinking. The most powerful tool is silence According to studies performed by The University of Groningen; brief silences in group conversations affect social needs. When the facilitator is silent or asks simple questions; participants often feel the need to ‘fill in the blanks’. Another upside, the UX team can conduct these interviews in rapid succession at our office, an inviting space for participants.
Video conferencing tools like Google Meet and Zoom enable researchers to connect to a global audience. Even tasks can be automated in Maze or advanced Figma prototypes, allowing for minimal interruption. Users will deliver a more candid and authentic response. But we’ve noticed that chances are more likely that users will exit the test without ever returning. By providing a sufficient reward at the end of the journey you’ll improve those odds.
The bottom line is that capturing the knowledge of these interviews and recordings is the most time consuming. Going with unmoderated recorded sessions with an AI transcript tool helps with that. You can use Loom or Bubbles to record, use Otter.ai or Granola to transcribe and one afternoon to summarise the findings.
When the facilitator is silent or asks simple questions; participants often feel the need to ‘fill in the blanks’.
Friendshopping vs Recruitment
To get a full unbiased look at your product, you might expect that going with an external recruitment bureau for finding usability testing participants is the only way forward. We’ve gone with UserInterviews in the past. They tap into multiple markets, have access to international users… Whether it’s NDA-locked projects, your new family brand or simply to experiment—they get things done quickly.
The counterpart is to weigh into your greater network by yourself.
We’ll be the first to note that friendshopping (seeking out users from within your network) can be just as effective. Even at the start of a UX Research project, your persona’s are always somewhat enforced by historical empathy; where the designer comes from, that’s where they start their process. Tapping exclusively into representatives that are similar to you is not advisable, as a diverse outlook on your product still helps it the most.
Assess what fits your own needs
Just four distinct types? It appears limiting. To broaden your view you’ll have to be cognisant of the available resources within your organization. Assessing and tweaking the type of testing according to your needs is a natural part of the process. Like, LEGO® blocks, you can always build unto your core and expand. If you have (1) you can then build it out by following (a,b,c,d).
If you have access to:
1. A team of creatives
You can utilise their brain-power to work in workshop/brainstorm/regenerative/retrospective formats, for example:
- Role playing as the user
- Brainstorm session amongst the team (Here’s a great result from a Stanford brainstorm)
- Make landing page improvements to test out your idea
2. Track analytics
You can track quantitative behavior on your domain(s) and see how keywords are interpreted by your visiting users. Furthermore, you can…
- Track user group movements on social media (platforms)
- Use heatmaps, surveys, click data
- Experiment with A/B versions
3. If you have access to a customer care team (internally or client-side)
You can set up interviews and cluster the returning topics together.
- Regular check-ins with customer care team
- Regular surveys with customer care team
- Approach callers through your team
- Set up a help-line specifically for your design
4. Host events
If you have the possibility to host people in a (physical) space, it can open up your doors in a variety of ways.
- Host event for stakeholders. This can very well be conducted via Discord, Twitch or Airmeet
- Get started inviting your network via Eventbrite
- Host a product launch event
- Invite larger groups of users for play testing session; a practise from the game industry that design has accustomed to.
- Your events are an exclusive maker’ club of people that improve the product over time
5. Workflow
If you have a say over your product’ or team roadmap you can sway the opinion towards more user testing… You don’t have to know upfront what questions you want answered, you just want to block in area’s for testing:
- Survey moment
- Guerrilla testing
- Finishing up prototypes
- Play test
- Host a hack-a-ton with your team
- Host a retrospective for your team (or with usability particiants)
6. Access to users
Whether it’s remote or not; access to users is not a luxury but a necessity. You might have a smaller pool due to your B2B service, your crypto/SaaS/fintech start-up hasn’t launched yet or your team is on holiday. But, just as your other business obstacles, they are part of an organisation with a multitude of contingency plans. If you don’t have access to users yet, you should start!
- Send out an email via newsletter to your subscribers
- Set up a community Discord server
- Engage with users on social media
- Set up a stand or shop on a local event your users visit
- Acquire email addresses via giving out freebies (ie. whitepaper, tutorial)
A cultural shift is necessary
Like Jakob Nielsen spells out in his article from 2005 (!) if you evangelise usability in your organisation, you’re more likely to change strategies as design maturity progresses. You’ll start with a strong foundation and continue onwards as your perspective on the matter changes and evolves.
Starting point: One or two people in the company care about usability, but working on usability activities is rarely their main job. As a result, they start small, typically by doing a little user testing on the side.
However, many organisations get off on the wrong starting point by using biased subjects (colleagues, stakeholders, investors) as participants. Or sharing only the solutions—without proper context—from their testing sessions with the design team.
“(This) frustrating reality is an open secret within the user experience industry and one we have long accepted as a normal consequence of working in a field that balances creativity and research” Tanya Snook writes for Fast Company. It’s easy to fall into the trap that is now being described as UX Theatre: the occurrence describes the lack of recognition for design- & usability testing as a skill. A perceived process that everyone can adapt to. Therefore the activities to validate design get trivialised. The truth of the matter is that an organisation, from Enterprise to start-up, has to ‘go for Context’ — because the user’s needs change over time. Whether an organisation (dis)likes the input, the final definition of done is kickstarted by user input. It stands as the most trustworthy source of information.
Introducing testing participants is not only like the hiring process; it’s also embedded in the critical design process itself. A similar practice would be ‘pair designing’—a type of partnership where designers collaborate with regular check-ins, and is loved by our Bolden team.
Using the thinking systems & expectations of pair designing with a usability testing participant is almost like having conversational check-ins with them. Instead of a design system, however, you both focus on user experience.
For Linn Harbo Dahle describes, UX-designer at Bekk, the core fundamentals of working in pairs as designers comes down to discussions as a healthy part of digital design.
If you get input and the discussion happens after you have landed on what you think is the best, that will not make a useful discussion!
Conclusion
To sum up the guide we’ve assembled this scope guide. Whether you need something today for a one time session or want to start doing tasks ‘from now on’ — here’s 20 activities you can take up right away.
In the design process, sure the KPI’s and USP’s can be pre-defined by management layers, but the user needs are best defined by the users. When stakeholders who are not directly using the product start deciding what’s best, time to get the expert advocates on board!
Involving the audience is truly taking ownership over the product’ outcome. Your product will not linger in groupthink but make valuable, long-term changes.
And it’s fun! There is no perfect way—it’s testing and trying and error and learning. It’s repetition. At the end of the day humans are fluid beings; strong opinions, loosely held.
