I just got back to work from Japan this past week, and I immediately had a user interview to sit in on — this post is a recap of what I learned while working directly with the UX research team.
I’ve been working on a project that’s meant to change one seemingly small aspect of our product. Initially, we thought it was going to be a short stint, one that would get mocked up and ready to go within two weeks. It has been nearly two months, and one of the reasons why it took so long (aside from resources, which, I assume, is the problem for many companies) is that we needed to go back and do some research. As usual, I have added a TL;DR at the bottom.
Ah, yes…dreaded research. As I mentioned in my last post, it is super tough to come to terms with being confident in your research, but you’ve got to do it.
First draft, second draft, third draft…of questions
In UXA, there really was just not enough time to come up with research participants, much less work on multiple drafts of the interview questions. I will tell you that it is absolutely worth the time, but from my perspective, it was near impossible to understand what questions were right.
Enter the UX research team. I’m lucky enough to work at a company with a separate research team, and they are all really cool people who are willing to teach. I sat down with the lead researcher, who had me draft up questions prior to the meeting.
I knew what I wanted to ask. For the sake of privacy and NDAs, I’ll give a general example of the sorts of things going through my mind as I wrote out my questions:
- Goal, Robbin Vision: We want to help people get to the teas they think are great when they have a bad experience with a brand of tea.
- Thought Process: Ask questions about the teas they like. Ask what they do when they have bad tea. Understand their reaction, then start asking about what we can do to help get them a better tea. Ask them for feedback on the prototype and see if we’re missing anything.
I thought it was pretty straightforward.
I was wrong.
In the meeting, our lead researcher took a look at my questions, and she said that it was a good start. (By this point, I’m feeling the color drain from my face, because I thought for sure that was comprehensive enough.) She gave me an excellent visual as she started stepping back from my extremely specific questions about the product and supplemented them with general questions about similar experiences that are not necessarily related to tea.
She gave me a quick sketch of an hourglass. “This is how I write my questions, usually — I’ll start with general questions about the bigger picture, get more specific, and zoom out so that their answers are more reflective of the product rather than that specific situation.”
So maybe in this case it’s what someone does when they’re thirsty. What do they look for in a drink? And then we’ll start asking what happens when a drink doesn’t meet their expectations. Follow that up (with awesome stealth and excellent conversation flow) with your specific questions about tea. Then bad tea. Then what they expect you or the beverage company to do about it. Take a step back and ask them about something more general — when is the last time you switched brands for something? Why did you change brands, and what made the other brand better? What expectations were met when you switched brands? What could the other brand have done better?
Whoa. Whoa. At this point, my mind was blown. It was such a beautifully crafted framework. Everything made sense and got tied together at the end. She coached me through writing the generalized questions and had me do another draft, this time, as a script. This was looked over by another UX researcher, who added further comments on wording, restructured when questions should be asked, and challenged me to think of what information we’d glean by the end of it all. When I got the OK on my final draft, I felt pleased, but also intrigued at how they had artfully turned my questions into something useful.
Watching the master work
I sat in on my first user interview when I got back from Japan. The PM and I were shadowing our most admirable UX researcher, who would be running the session on Skype. We were to stay quiet and send over any follow up questions we wanted her to ask over Slack.
Yeah, okay, it was pretty neat to hear my questions getting used, but the real source of amazement was listening to how the user answered those questions. He gave thoughtful answers that all flowed well together as she asked follow up questions. Since it was a live session (and it was about his experience with bad tea), he felt a little more sympathetic toward the bad rating he gave and further explained what he probably would have done. It was freakin’ amazing.
It took all my willpower to keep my jaw in tact, because I was blown away. All that? From a 20 minute session? UX RESEARCH IS RIDICULOUSLY AWESOME.
The interview recap
We had a 10 minute debrief after the session. We learned that the user actually expressed different intentions — he didn’t care about getting a refund. He just wanted the better brand of tea, no matter how he got it. It turned out that our hypothesis was pretty much off the mark.
The UX researcher had also taken notes, and we had a Google Drive folder with all the information we got from our six interviews. It was like seeing something blurry come into focus.
We learned some surprising things during the interview, and our PM was able to express what he wanted from the product design team (sort of a UX generalist thing for us), what he had to learn from engineering, and how long it would take to implement, if it was all possible.
Now we’re looking at a potential ship date around mid-December (fingers crossed) and I’m over the moon.
- Write questions in an hourglass format. Start with general questions about the user’s habits. Zoom in and get to the specifics of what you want to learn. Zoom out so that they’re not only thinking of those specific topics (to see if their answers to your specifics matches what they’d do in general). End with a general comments/question part.
- Write a script. I think we’re meant to do that anyway in UXA. Keep it conversational. Listen to this Mixed Methods podcast episode on building rapport with the person you’re interviewing. Remember that people in general actually like talking about their experiences.
- If you are sitting in on an interview, do so in a way where you’re not visible. It’s more to keep the interviewee relaxed. Do YOU like answering questions when more than one pair of eyes are on you? Nah, man.
- Take good notes. If you had folks also sitting in, compare them. It’s interesting to see what the takeaways are.
- If you’re testing a prototype, make sure it’s ready and have a separate set of questions for them. We already do this in UXA, but it helps to ask things like, “What would you say is missing / if you had a magic wand, what would you want to see here?” at the end. I know it’s a no-brainer, but we didn’t have the prototype ready yet when I was first writing the questions.
- Find the patterns and share with the designers. After 3–5 interviews, depending on your topic, some patterns or interesting behaviors may have emerged. This is the stuff that will be given to the UX/product/etc. team, who will then take that research and iterate on prototypes or start wireframing, etc.
All right, y’all. I hope that was insightful. As a disclaimer, these are, of course, just my thoughts on how it went. It was a great experience for me, so if you ever have a chance to shadow someone in the field of UX you’re interested in, DO IT! Act now! Remember that learning can’t just happen in a bubble.