Are Your User-Tests Too Clinical?

I have the privilege to be the UX designer at Thalmic Labs, the company behind the Myo armband. It’s a blast developing for that little piece of engineering brilliance. One thing we try to do at Thalmic is gather a lot of ideas, talk about concepts, and concentrate on the ones that seem to make the most sense. Then we invite people who are not part of the company to test out the ideas.

Testing your product with people (user-testing) is necessary to gain insight on how certain aspects are actually perceived. We might think that “A” is a great idea as we talk about it on a conceptual level, but it might test very badly. I like to think of it this way; testing is like opening the windows and letting the fresh breeze of reality liven up the office.

But of course you test your product, that’s a given, who doesn’t? This post isn’t about making a case for testing, we’re beyond that. What I’m interested in talking about now is the environment you’re testing in.

Ok, what should I do now? — Merp…

I can recall a discussion with a colleague about scripting user-tests. He suggested that we should be very intentional about offering a neutral and consistent baseline test experience for each participant, because it appears to be the most scientific way of gathering data. For a while at Thalmic we took this approach and it felt good to us. We would script, rehearse, and be as prepared as possible.

The reality seemed to be, however, that the unusual nature of our rigid interactions caused people to be further removed from a situation where they would use our product.

Does this sound like you? Can you recall a participants’ reaction when they asked you a question and you paused to sift through your script for the appropriate response?

Tests where our effort was spent on having neutral responses seemed to put more emphasis on ensuring the test was correct, rather than allowing an organic meeting to listen and learn. Mechanical interactions don’t create a situation conducive for people to express themselves comfortably. It might as well be an online survey.

More life; less laboratories

Think about it. When someone uses your product in real-life they are likely to be in a familiar environment with a certain degree of distraction. I used to have this notion that when people would use the Myo Getting Started Guide, they would go into a quiet corner, shush their kids and put their devices on airplane mode so they could give it their undivided attention.

I realized however, that when I use a new product for the first time I’m excited and, heck, I’ll even go through a setup for the first time while driving. I’m willing to endure any distraction, I just want to dive in. We should expect no less from our real-life customers.

Testing laboratories would ideally be more familiar as well. If your lab is a desk in the middle of a white-walled, undecorated room, don’t expect people to give open and honest responses.

Your goal should be authenticity

If you want real and honest responses, simulate real life: Let conversation be open, make your testing lab feel more like a room in a grandparents’ home.

Then throw paper airplanes at them.