Never Let A Designer Do Their Own Usability Testing

They’re different skills

At a recent meeting for the design of an app a client asked if we offer usability testing as a distinct service from our design process. I answered yes, but asked why she wanted to use separate vendors.

She replied with:
 “In the past we engaged with a firm for design and usability testing. The product came out of testing with ‘no reported usability issues’ but when we launched it we got tons of horrible feedback from users. We later found out that the ‘UX Specialist’ did their own testing.”

This client got burned and — unfortunately — it’s not a unique case. There are many firms where the research, design, and testing is all done by the same person. Or, even worse, when they claim to be able to ‘do UX’ yet they brand themselves a full service digital agency. In this landscape it’s natural to want to use different vendors after such an experience.

Design and usability by the same person is a flawed way of approaching product design

1. Research and design are very different fields and its rare to find one person with the skills and experience needed to be exceptional at both. They exist, but they’re a rare breed. Good testing moderators probe without leading the user so you can actually uncover usability issues and actionable insights. This is a difficult skill that requires a lot of practice on the part of the user researcher.

2. Design is an ego driven profession. A designer cannot be impartial with their own project. Hearing users critique the weeks or even months of your effort can be uncomfortable — particularly for new designers. It’s difficult to remain objective. It’s too easy for a designer to say “oh that’s the users fault that they didn’t see it. It’s right there, they should have looked closer”.

Using a third party consultancy eliminates the need to keep a researcher or a dedicated usability specialist on staff at all times. Additionally, having usability information presented by an impartial third party can add to the credibility of the findings, and keeps internal organizational politics out of the equation.

Using multiple vendors for product design can get very messy — very quickly

Good design and good products are born from good communication. Transferring insights gained from one company to another isn’t easy. When it’s the same vendor, the designer and researcher can collaborate and communicate naturally. Swivel chair benefits! You can’t share insights effortlessly when you’re not sitting in the same office.

Vendors might not play nicely together — they are on different teams after all. Each firm will have their own agenda that they’re working towards and if there is an overlap in service offerings, these agendas will be at odds. This benefits no one, least of all the client.

So, while I sympathize with the client’s situation it’s not necessarily the best way to design a product. Finding a single vendor that you trust with individuals or teams who have all the necessary skills and experience is a far less risky option for you as a client.

Originally published at on September 21, 2016.