Some Lessons Learned Doing UX Research
Methods and tools aren’t the answer.
When I moved into UX research at ResIM a little over a year ago it was cool to hate focus groups and surveys. It’s probably true that they’re used too often, or too often used instead of more appropriate methods. But I’ve found focus group-type sessions to be great for generating a lot of input quickly, and surveys are great for validating initial findings across a larger sample.
People love to talk about methods and tools in any profession, but sometimes we seem to treat methods and tools like they’re the answers and solutions themselves. We have a very broad and rich landscape of UX research methods to work with, each suited to different goals and questions. And each method can be applied a multitude of ways…
Methods and tools matter, but not as much as actually doing the work, asking the right questions, listening, being critical and exercising good judgement. Sometimes you can do all that just by leaving the building and talking to the first five people you see.
Small insights make big differences.
For me, this has probably been the most interesting and unexpected lesson learned doing UX research.
All of our clients have had good ideas and insights before we’re involved. Some have had great ideas, especially when they have their own internal researchers and designers. If we compared snapshots of the lists of ideas before and after our research process they’d often look nearly identical.
But “ideas” are often where people get stuck. Research grounds ideas in reality to give them traction to move forward. What really changes through the research process isn’t the ideas themselves so much as the degree of clarity, understanding and alignment around them.
When done well, UX research turns hypothetical visions into stories that move everyone in the same direction, with more urgency and a clearer sense of why and how.
Collaboration is everything.
If the purpose of research is discovery and learning, then it’s important that we learn together. I’m not adding much value if I spend weeks learning what someone could’ve told me at the kickoff meeting. Research should build on, refine, and share what people already know.
Likewise, my value is pretty limited if I dump 50–100 pages of findings and recommendations into a report and move on. It’s too much to digest, especially if the questions the report answers aren’t top of mind the day or week it’s delivered. People go back to what they’re doing and forget which proverbial shelf the research is sitting on.
The delivery of a report is just one milestone in a process of interpreting results and executing on recommendations — many of which are best formed later as more granular questions and constraints emerge. There’s just no way to anticipate every execution-related question that could emerge. Research needs to be responsive to questions.
Of course I already “knew” this by reading books like Just Enough Research and Lean UX, but my appreciation for collaboration has deepened through experience.
Knowledge is both useful and dangerous.
This is a lesson I learned long ago that continues to be reinforced every day. Knowledge makes us feel smarter than we are.
On every project I talk to someone who says “everything needs to be within 2–3 clicks.” It’s a widely debunked UX myth that people still cling to. It’s not the only one. If we’re not careful, rules are handy ways to unconsciously reinforce biases—whether it’s a stakeholder who wants visibility for their department on the home page or even another designer citing principles that align more with their past experience than the current situation.
Look at how assumptions about things like flat design or hamburger menus have fluctuated. Rules and principles intended to address a specific place and time easily get repeated as misconceptions in other contexts. If psychology can’t take its past findings for granted, then why would UX research be any different?
“The days when a researcher could observe a few folks using a website on a desktop computer, crank out an observation like “users hate carousels” and call it a day are so far behind us it’s difficult to make them out in the rearview mirror.” — Matt Gallivan (Airbnb), Embracing Uncertainty in UX Research
A better approach is to treat knowledge as living system that continuously evolves or grows from one context to the next—from print to digital, or from web to mobile, wearables, AR/VR, AI and chatbots, etc. Even as I’m writing this I’m trying to be mindful that these lessons learned might have a limited shelf life. The best I can hope is that they lead to better lessons.
It’s often better to be wrong than vague, so general principles are a good baseline to start from—as long as you stay critical and observant. This is how science works: it’s not a solid edifice built one brick (or principle) at a time, it’s a process of continuous discovery and reformation.
