Great User Research (for Non-Researchers) — Part 2
Doing Research
Summary
The demand for User Research exceeds the supply of researchers. And when, ultimately, our goal is “Learning from customers’’ and our user research community is a helping profession, we should be finding ways to empower everyone to do well. These three connected essays aim to help overcome territorial and quality concerns within our community. In this second essay, I provide some of the essentials on doing user research for those “other people” wanting to go out and work with users. The previous article was about planning research, and the final piece looks at acting on user research.
Running Interviews
As a user-research technique, interviews may appear misleadingly straightforward. Since it’s really just talking to people, what’s the big deal? Of course, there are types of work that is just talking to people (such as hostage negotiator, therapist, or journalist) but, of course, we know the practitioner’s approach to what to say, when to say it, how to say it, what not to say, and so on are specifically selected to best achieve their professional aims. Much of what we practice has roots in social science, but examples of similar best practices can be found in many different contexts.
In an interview, specific questions are better than general questions. “How do you typically choose a contractor?” is a hard question to answer as it presumes that the participant has a set of practices in place; if they don’t, they may make something up to address your question. Instead, ask about the last time they started a project, and how they chose a contractor, and then ask about whether or not that represents a pattern. It’s easier for the person to give you a more reliable answer that way. When Malcolm Gladwell interviewed Pentatonix for his podcast Broken Record, he demonstrated two extensions to this principal. First, when people give a vague answer like “It doesn’t work for me” he follows up by asking for a specific example. Second, when people describe a process vaguely, he asks them to show him that process. Ask follow-up questions to uncover the specifics when you get general answers to your initial question.
Don’t Put Answers In Questions — A bad practice that many people default to is putting the answers in the question. The good question is “What happens to the package when it arrives?” The bad practice is to say “What happens to the package when it arrives, do you open it, or hand it to someone else, errr…” This is a really hard habit to break and if you think that the participant can simply say “Well, it’s none of those, actually”, you underestimate how much power you have when doing research. Eventually, a participant will reflect back the framework you give them because they want to do a good job. This is hard to stop doing — so keep practicing this.
Hold Off On “OMG, I’m the same!” — The naïve approach to building rapport with someone you are interviewing is thinking that you can connect with someone by telling them the ways that you are just like them. That takes focus away from the participant. The best practice is to hold off on that reaction and keep asking them questions. That focus on them is what builds rapport. You can very occasionally and calmly reveal your own perspective or experience only if you need to normalize something they are uncomfortable about.
Deflect Roadmap Questions — If participants start asking questions like, “Will the next version have the search console integrated?”, do not answer them. Once you do that, you become the expert and it’s almost impossible to return to research mode, where they are the expert and you are interested and curious. You can use the standard researcher deflection and turn it around with “Why is that important to you?” Or, if that is too confrontational, you can say it like this, “I’m going to act like a researcher and turn it around to ask ‘Why is that important to you?’”.
Hold Advice Until End (if at all) — In addition to people mispronouncing the name of your company, you’ll also hear them wish for features that actually do exist. Do not interrupt the interview to explain your product to them. Keep quiet, no matter how hard it is. Otherwise, you are making yourself the expert. At the end, when you are all done, you can tell them anything that would help them, but this isn’t a chance to evangelize your new feature or encourage them to do what you are hoping they would do, it’s only for something they need help with. So do it very sensitively, “you mentioned you were looking for video tutorials, would you like me to show you some?”
Speak Their Language — Social scientists talk about “natural language” — the way people in a certain group refer to things. Sometimes local terminology may vary from yours. As you learn these terms from participants, reflect that language in the way you word your questions. Don’t try to be cool and presume shorthand that they haven’t shared with you. For instance, don’t respond to a mention of “National Geographic” with “NatGeo”; instead use the language they use. This reinforces that they are the expert and that you are curious about their perspectives.
Questions Will Generate Stories — It can take a while until you establish a sense of rapport, of comfortable back and forth, with someone you are doing research with. It depends on you, and that person — it might take 10 seconds or 10 minutes or 45 minutes. You have to be patient. Consider that over time, the interview moves from question-answer/question-answer to question-story. Just because that person isn’t effusive right away, give them a chance to get comfortable with you, and with being asked questions about the topic. Don’t let yourself get too uncomfortable if the interview feels awkward for a while, just keep asking questions.
Most Questions Are Follow-Ups — You should definitely have a list of questions prepared, but most of the questions you actually ask, and the way you ask them, should be emergent — they should come from what the person has said. You’ll need to follow up to get clarity and additional detail until you get to the point where you understand what they are trying to tell you. When you go back to something from before, tell them “I want to go back to something you said before.” When you change topics, just tell them that you’re changing topics with a phrase like “Okay, I’m going to switch gears here and maybe we can talk about planning out budgets now.” This reinforces that you are paying attention to what they are telling you and it’s important and you want to know more about it, or that you are partnering with them in this conversation and you are making sure they’re tracking with you where it’s headed. This really increases rapport.
Don’t Just Show Your Designs — When it comes to showing prototypes, I find a lot of teams default to taking their design process artifacts and testing them. This is missing the opportunity to create new things to show in the field things that are not part of what you’re planning to build, but are tangible, experiential, visible artifacts that people can play with, or react to, as a way to provoke a deeper reflection on the underlying issues that you want to understand.
Beware of Satisficing — Below is a stealth photograph I took on a plane where the people in front of me had rigged up an iPhone holder using the plastic bag from the blanket. They made a sling and jammed into the tray table hook. This is a great example of a “satisfice” — or a “good enough” solution. Satisfice comes from the words “satisfy” and “suffice” and it’s an important concept when doing research: people have a high tolerance for good enough solutions, and when we see satisficing in the field we sometimes decide it’s an opportunity. Sometimes it is; some aircraft interiors now have clips on the seat back where you can mount your device and watch during the flight. And sometimes it isn’t; people will continue to put up with a non-ideal solution.
You can think about the cost of tolerating an existing compromise versus the cost — the financial cost, the cognitive cost, the switching cost — of adopting a new solution. If the cost is zero, like it’s in the seat-back already, then sure. If the cost is higher, like it’s something the traveler has to learn about, purchase, pack, and carry, then maybe this is just fine for them. So you’re thinking about the behavior change required, not just rolling out new fixes because you can. Those will not be adopted. Try to see what people are doing as something that works well enough for them.
Make a Recording — Just mechanically, given how fast people talk and how fast you can type or write, you can’t take notes well enough. You need a recording that is a complete and accurate document of the interview. Notes are biased — it’s what you heard and interpreted. You can take notes as a way to help you process information in real time, jotting things down, marginal notes, quick thoughts, etc., but get a recording for the official capture.
Biases — People get stressed about doing research because of general uncertainty and fear of biases. There are so many of them and unfortunately, a great deal of bias is just how our brains work. My advice is to work on hearing your own bias, when you have moments of being wrong, you can reframe it so that uncovering your own judgment is joyful, freeing, and liberating. You go from “I was wrong” to “I am learning.” You want to apply this approach to any moment in a research session, not just feedback about your product or design solution. And of course, this is a skill to develop for your life in general, to just practice being excited by being wrong about something.
Empathy and Judgment — Empathy is a word we talk about so much, and in our lives and in our profession, empathy is important. But for research, it’s specifically important because it’s how you do the work — understanding the contours of someone else’s perspective and experience when it differs from yours is exactly what it takes to do user research. That is what you are trying to learn in the first place. So being judgmental, which is a very human thing, gets in the way. We all do it. What did you think when you saw that plastic bag as an iPhone sling? What will you think when you see that from one of your participants?
To confront our own judgment, first learn to hear yourself making those judgments. Then practice simply questioning your assertions. Maybe that isn’t true. If you can, get more data to flesh out a new conclusion. Even if you don’t, I believe that challenging your judgment is a way to open yourself up to the kind of revelations we’re seeking. These two emotions, empathy and judgment are tied together, they are in adjacent regions in the brain. And that enables an amazing life hack for when you are feeling judgmental and angry; you can do this any time, it doesn’t have to be in user research.
For example, I was traveling recently and had a long morning walk to go meet my friends and I was late, hurrying, and generally feeling grumpy. I cut through a park and there were a bunch of people getting off of tour buses, crowding around, really getting in my way. I felt annoyed and critical of them, but then I strode past a three-generation family, where the mom was posing people to take a picture. I happened to make eye contact and gestured that I could take a picture of all of them, and so wordlessly they gave me the camera and posed for me while I took a vertical and horizontal picture. I handed the camera back and they expressed so much gratitude. The thing here that I felt so much better. Beyond being a good person in the world, extending empathy is good for us — we feel better.
Finally, there’s brain science that tells us that stress limits our empathy. That means you should manage research project schedules so that the workload is reasonable. Don’t schedule too many sessions in a day. Make sure you have breaks. Make sure you plan for whatever might stress you out: logistics, technical details with your equipment, driving directions, and so on.
Since interviews resemble a normal conversational interaction, it can take a lot of practice to unlearn one’s default, verbal habits and develop these best practices. Play back your own interviews to take note of where you could have done better and you will be more mindful in future interviews.
In the third article in this series, we consider what to do with the data you’ve gathered — acting on user research. Also see the first article — Planning Research.
