Sometime people ask me questions about UX Research, and sometimes I’m able to give them good answers on-the-fly. For all other times, I’ve created this FAQ.
Q: Gathering feedback seems time consuming and a waste. People don’t know what they want anyways — didn’t Steve Jobs say that?
There is a big difference between running a useless survey and actually putting in the work of gathering feedback on a design or new tool.
At the absolute minimum, gathering feedback can help you from making mistakes like this…
And if you have a UX Researcher helping you out, they will be thinking about the longevity of the product and prevent you from making mistakes like this…
Lastly, if you invoke Steve Jobs to justify your reluctance to do research then you are just trying to get out of doing work in the lamest way possible. (And you should read this: That Steve Jobs Research Quote Should RIP)
Q: Doesn’t research cost lots of money?
No. UX research actually saves your company tons of money.
It is way more costly to spend engineering time and resources on an idea that is flawed than to spend time in research getting the details right first.
Q: What is the best organization structure for UX Research on product teams? Who should they report to?
I’ve seen the org-chart for UX Research done one of two ways:
- Internal research consultancy — a small team of researchers report to a “lead researcher”. They receive research requests from all different parts of the company. They prioritize those requests and assign researchers as jobs start and end. Researchers get a wide-variety of experiences working on a wide-variety of products.
- Embedded on product teams — each researcher is assigned to a single product team. They become experts on their product and target customers. They report to the team product manager. (But may also have a “dotted line” research manager.)
Typically I see “structure one” at small companies that have few resources and/or not enough work on one product to occupy a researcher full-time. I currently work at a company that operates like this. There are only a handful of researchers and we take on “jobs” from all different parts of the company. Our “customers” include sales, operations, engineering, and product.
“Structure two” is what you see at bigger companies, like Google and Amazon. Researchers are deeply embedded in a particular product domain. They may collaborate and share ideas with other researchers, but typically their day-to-day interactions are with the product managers and engineers on their team.
(Have you seen any different kind of structures for UX Research? I’d love to hear about it!)
Q: Don’t you need a control group and a treatment group and then run a statistical test to really get a valid answer?
The shortest answer: No. You don’t.
The slightly longer answer: Quantitative research methods are appropriate and valid in some situations. Qualitative research methods are valid in others. Good UX researchers have studied and trained in a variety of different methodologies (quantitative and qualitative) and are skilled at selecting the right method for the particular research question.
The longest answer: Look, I know that you took science classes in high-school that have biased you into thinking that science is really only done when you have a hypothesis and an A/B test. I also know that they have biased you into thinking that looking at “anecdotal evidence” is a cardinal sin. And to you, qualitative research looks a lot like anecdotes and this is giving you the heebie-jeebies. Let me put you at ease.
First, a point: the foundation of all science is observation and documentation. The Theory of Evolution is based on observation and documentation. The Law of Gravity grew from initial observations, which were documented. Everything we know about animal behavior, from their migration patterns to mating behaviors, is based on observation and documentation. Science = observation and documentation. So whenever I do anything in my job, I always observe and document. It is the absolute minimum requirement to doing science.
Even in “quantitative research” you observe the experiment and document everything you see. This is because the world is inherently uncontrollable (especially when you deal with complex animals like humans). When you observe and document in a quantitative study, you are looking for a couple things, like threats to the validity of the study, and further explanation as to why the treatment worked (or didn’t work as the case may be).
When you observe and document in a qualitative studies, on the other hand, you are uncovering “themes.” Themes are, in a nut-shell, the most common behaviors you see when you observe people interacting in a specific scenario. For example, when you conduct a usability test you are trying to document the most-common behaviors, problems, and interaction flows participants have with a specific interface. There is a lot of history, science, and tradition within the fields of anthropology and sociology that UX Researchers rely on to conduct these kind of tests.
Add more to my FAQ!
If you have questions about UX Research — ask them and I’ll try to answer.
If you are a UX Researcher, what questions do you routinely get asked?