The UX research method choice issue(s)

A short survey summary and some good news

A while back I ran a short survey asking all sorts of UX people (from Facebook and LinkedIn UX-related groups) some questions about their UX research method choices. Over the course of two weeks I’ve gathered a decent 106 responses* and they actually tell an interesting (albeit limited) story, as well as point out some of the problems we’re all facing in our day-to-day work. The reason for running the survey was simple: I have an idea for simple tool that could make the research method choice a bit easier for all of us, I just wasn’t sure whether anyone would actually need the tool.

The questions were designed to gather a wide view of the problems related to conducting research. About 3/4 of responses were from non-researchers and those people I asked two additional questions:

Do you have access to a full-time UX researcher or budget for external research?
Do you sometimes organize and conduct user research yourself?

To my surprise, both those questions shown a quite strong trend among the respondents. About 2/3 of them don’t have access or budget for a researcher and a mind-blowing 90% actually conduct their own research. What that tells me is that there’s a bigger group of people who could potentially benefit from the tool that I initially anticipated.

What is more, the ‘tie breaker’ question shown a significant interest in the vaguely described tool:

How interested would you be in a tool to help you decide on a research method that best suits your questions?

With almost 88% of people either interested or strongly interested in the tool, I’d say it’s enough to hypothesize that there’s definitely a market for such solution.

Next I asked everyone more generic questions:

How often are you confident that the research you’re conducting is answering your questions in a reliable and actionable way?

Only about 20% of people are always sure about their research findings with an overwhelming 70% checking the option sometimes, which tells us that there’s quite often a bit of doubt in our own research. Why might that be? Are we just being overly cautious (or maybe just the right amount of cautious)? Or maybe it’s the effect of majority of people not having access to researcher?

Last question was about pain points that arise when deciding on a research method and here are the 4 most interesting takeaways:

  1. Over 50% of people don’t know which method would best suit their specific needs.
  2. Exactly 50% of people are having trouble finding the right participants.
  3. About 35% of people don’t know how to get participants at all.
  4. Slightly over 30% of people don’t know when is the right time to conduct research.

All of the above might potentially point to the results trust issue I mentioned before (if I’m not sure about the research I’m conducting, obviously I won’t be really trusting of its results), therefore they are all worth addressing. If you have an idea on how: go for it! There are people waiting for you to make their world a better place (about 93.4% of people to be exact, since only 6.6% of them checked the ‘I have no pain points’ option)

There’s actually one more issue that I didn’t anticipate at all but came up a lot in the open-ended questions:

I’m having trouble convince product owners and their superiors that user research is needed and profitable
Lack of budget for research and openness, willingness of steakholders to implement them
Hard to get stakeholders on board. Funding and timeline constraints always an issue.
Have limited resources and time so compromises need to be made
A lot of times it’s an issue of time, budget, and access to participants. I work at a startup with no funding, so budget is big on my mind (…)

Now that’s a big one and one that was — as you can see — mentioned repeatedly. I can’t say I have a good answer for this one, but I now realize that in some cases this would be the main thing stopping us from doing our job the right way — which is by basing our decisions on data. Maybe promoting methods that are less money- and time-consuming could move the needle for this one? Again, if you have an idea on how to fix it for all of us, feel free to chime in, I’m sure we’ll all be grateful for this one :)

Now for the good news: based on the survey results we’re now starting to work on that research method selection tool! It’ll be aimed mostly at designers and Product Managers/Owners but we’ll try to make it available and usable by everyone who’s taking part in any product-creation process. We’ll also make sure it’s free of research lingo and understandable even for less experienced folks.

That’s it for today, thanks a lot for reading this short summary and hopefully I’ll be back here soon to unveil the tool! In the meantime though — do you have some other problems related to UX research that I didn’t find in my survey? If so, please share here, let’s continue the conversation and think on ways to fix those!

*One thing to remember is that the results are most likely a bit biased (I can’t really check how much) towards Polish UX-people. The biggest trends didn’t change with more international responses though.