The dangers of UX research…

Dawn Armfield — https://unsplash.com/photos/28v9cq7ytNU

Cars are dangerous in certain contexts…

Knowing when they are dangerous is important, and knowing when you should or shouldn’t drive is also important. If a driver is drunk, texting, or just inexperienced the car becomes dangerous.

We mitigate this danger by communicating the dangers of cars when put in the wrong hands. The messaging around drunk driving is clear and severe — it kills people. Same with texting while driving.

Very clear issues with very clear results.

The point of this is not to educate you about driving dangers, but to point out that…

People gain clarity through clear communication…

If the message I send is cloudy, the communication is cloudy. And the message I get back is likely going to be cloudy as well.

So far the user experience community has done a pretty crap job of educating people outside the community about what we do. Mostly that’s because we’re huge nerds about things like P-Values, time-on-task, new books on research methodologies, and things most people don’t care a bunch about.

We try to explain this stuff and show how it works but normal non-nerds eyes glaze over and it’s difficult to force someone to watch research being done… (unless we apply the Ludovico Technique, that is).

The point of this is not that we can’t communicate, but that we all need to remember…

Many things seem deceptively simple…

It looks like “researchers” are putting on airs and trying to seem super important. Yeah we all like to be important but the point is that if you don’t know what you’re doing, you’ll probably screw it up.

If I tried to repair my (imaginary) Tesla I’d probably screw it up because I don’t know what I’m doing. It’s very clear that fiddling with high tech devices is not simple, and is a bad idea unless you know what you are doing.

It’s less clearly communicated that doing crappy research will lead you to crappy results. It’s just asking questions, and it seems so simple…

I found this in Slack without source attribution, unfortunately.

This isn’t to say that you can’t do clear research, but that…

Clear research comes from clear communication…

Asking clear questions seems simple, until you try to avoid influencing the respondents.

There’s a not-very-nice joke floating around out there where the joker asks the victim something along the lines of, “have you stopped beating your dog yet?” Any way the victim answers it reflects poorly on them.

This is a leading question, which means it leads the audience and the victim to draw an unrepresentative and unflattering picture of the victim in their mind. This is just one of the many ways to influence how people respond to questions.

This is not to give you a way of asking mean questions, but to remind you that…

Doing flawed research is deceptively easy…

So much of what researchers do appears easy. Writing down questions, picking people to ask, and asking the questions seems so simple, but there are many ways to do bad research.

Here is an incomplete list of things to avoid when doing research:
* asking the wrong questions
* asking leading questions
* asking the wrong people
* asking about the future
* not talking to enough people
* focusing on confirmation bias

With one or two of these issues present your research results will be biased. If three or four issues show up regularly you’re likely to have substantially skewed results, and if you are just writing down questions and asking away it’s very likely you’re going to get information that is total crap.

This isn’t to say top-notch research is impossible to attain, but it’s important to realize…

Great researchers produce great research results…

You wouldn’t hire a firefighter to teach trigonometry or a trig teacher to fight fires. Experts niche down into doing what they do very well. It doesn’t make sense to expect user experience research to be any different.

Finding an expert who is able to put together an unbiased research script is not very difficult. Learn a little about the things above and ask questions about how the researcher controls for them. If they can answer clearly and don’t seem flustered you should be fine.

Asking for their assistance in recruiting and interviewing isn’t hard either.

Research is fun, easy, and yields great results when done well. But doing it well requires learning, experience, effort, and practice.

And being a huge nerd.