Testing your product with a diverse audience

Over the past 18 months, the work in our product team at CHOICE has been better as the result of consciously integrating diversity into our UX practice. In addition to the primary user’s needs ( e.g. ‘looking to buy household appliances’ or ‘consider private health insurance’) we also asked our recruiters to recruit people with diverse backgrounds and characteristics.

“But personas are not meant to be demographic!”

Yes, comrade, I hear you. Users’ goals and motivations for the product should still be your primary criterion and diversity second. Recruiting diverse users can also help you demonstrate how common behaviours occur across different demographics and work in your favour.

“Why should you care about diversity in testing?”

Build empathy

Having a face-to-face conversation with someone often opens up a new level of understanding and connection. All of this matters and enriches the final solution you’ll come up with.

In a recent user testing round for the new design of our product reviews, we had a user who had recently moved to Australia with her partner who’s not so fluent in English. Quite quickly, we could see she really struggled to understand what our company does from the test wireframes alone.

She thought we were an online shopping site for whitegoods, but that she had to pay for membership in order to buy from the site (in fact, CHOICE members get access to independent, expert reviews, funded by their membership fees, and we don’t sell products). Frustrated and confused by this rather strange business model, she said she would leave the site at that point. Yet in the previous session, another user who is a native speaker had been complimenting the tongue-in-cheek copy.

Watching this woman’s reaction to the text and design was powerful. It led us to make the design less like a shopping site and make the copy much more straightforward and easy to understand.

See a range of perspectives

By testing with people from mixed backgrounds you’ll get a more well-rounded view of their different abilities, cultural biases and personal contexts that will have an impact on how they use the product in real life.

In an early iteration of our health insurance comparison tool, Adam, my UX colleague at CHOICE, had text on a dark background banner on the page. An Asian tester pointed out to him it’s not a great idea to see black on a health-related site, at least for people like her. In her culture, black symbolises bad luck and funerals… all the things you don’t want to think about when you pick health insurance.

Get stakeholders’ buy-in

While there is data showing how you can find pretty much most of the usability issues with just five users, there are always times when a stakeholder will be sceptical of the small sample. Adding diversity dimensions to the small pool of users you test with face-to-face will increase the chance of uncovering more issues. It’s also easier to get stakeholders’ buy-in with a richer sample, rather than findings from a few homogenous users.

“How do I add diversity to my UX practice?”

Dos:

• Add diversity to your user recruitment brief: aiming for a good mix of age, gender, ethnicity, income, socio-economic background could be a good start.
• Be inclusive: think about people with vision, hearing and cognitive disabilities, as well as older users, those from non-English-speaking backgrounds, and people with varying levels of education and literacy.

Don’ts:

• Don’t include diversity for the sake of it, the target still needs to be relevant to problems you’re trying to solve.
• Don’t dwell on the difference or peculiarities of certain groups.
• Don’t expect diverse users to turn your design upside down. They might, they might not. Sometimes it’s about designing for what we have in common rather than what differentiates us.