User Testing is a Waste of Money*

It was four days to launch, and all the features were locked. The dev team was in heavy QA mode, and the whole team was on edge. I myself was madly logging bugs when the product owner sidled over to my desk and asked a seemingly obvious question. Almost rhetorical.

“Do you think we should user test this?”

I gaped. I’m sure I looked like a landed trout for a second. “Uh…what for?”

The truth is, we had been busy heads down building features for months. Requests to conduct tests during early phases were weighed against the looming launch deadline. At every decision point, the call was made that testing would have to wait. We were to move ahead using our expertise and assumptions. “Just use your UX best practices.”

Testing once all the features were built was going to reveal a bunch of issues we couldn’t afford to fix. Maybe even might uncover a pivot we could no longer pull off.


Sound familiar?

Testing at the end of the development process (aka “user acceptance testing”) is an outdated practice that puts the people who use (and pay for) software at the bottom of the value stack. Don’t do this. Be aggressive about proving out ideas quickly, scientifically and cost effectively.

If as a business you want to increase risk and drive the cost of change as high as possible — wait until a product is built before you test it. It’s madness.

*user testing is often done after important decisions and great expense. What a total waste.

Here is how to maximize your UX investment with a (smarter) approach that isn’t a waste of resources.


3 Tips for Effective User Testing

User testing can have an outsized impact on the future success of your company and product. In order to maximize the effectiveness of everyone’s efforts (and by implication, overall product spend), here are some things to think about.

  1. Formative vs. Summative testing. To reduce the cost of building software, testing should be done most often before development starts, while product features are in formation. It’s easy to change things at this point. Summative testing (after the fact) typically highlights issues that are already built into the product and are exponentially more expensive to change or fix.
  2. Part and parcel of Agile. (or whatever process you’re using). User testing should be part of the daily routine of design work, and ingrained into the task templates used by the team. User testing as a habit — rather than an exception — is the best insurance against opinion-based design and “developer myopia” (a cognitive blindness that comes with close proximity to a product or domain).
  3. Learning density > feature density. User testing must be done in a way to maximize insight, rather than maximize speed of feature development. Building a single, killer feature with pinpoint accuracy on a user need, is much more profitable to a business than a collection of features that accomplish the job in clumsy fashion.

Does your team do user testing? What works for you? Leave a comment!
If you need help putting UX to work in your organization, give me a shout :)