Uh-Oh, My UX Research Participant Is Very, Very High
4 lessons from facilitating a UX testing session with a participant who’d just sparked up a joint
“So where do you think that [link] will take you if you click on it?” It was the kind of probing question I’ve asked a hundred times. But I never got a response like the one this participant — let’s call him Jerry — gave me.
“To another dimension,” Jerry chuckled. I don’t think he was entirely joking.
Around 20 minutes earlier, when Jerry had joined the Zoom call, he turned on his webcam to reveal a dark room in which a constellation of rainbow DJ lights danced playfully on the ceiling. In one hand he held a cocktail — served in what appeared to be a pineapple mug — in the other a joint.
I have to hand it to Jerry. The dude was clearly living his best quarantine life.
This was not going to be a typical qualitative UX interview. Indeed, at one point Jerry brought the website-focused chitchat to a halt so that he could tell me how much he was enjoying the sound of a bird call in my background noise. To him, the distant squawking had conjured images of palm trees and sunny beaches. (Alas, I was not conducting my session on a Caribbean island as a parrot breezed by; the bird was a garrulous crow in rural Vermont.)
When I realized that Jerry was very, very high, I initially considered finding some excuse to cancel the session before it started. I had reservations about the validity and sheer usefulness of the data I’d be able to collect from a participant under the influence. However, I’m glad I didn’t balk at the prospect on the false assumption that a participant who’d just sparked one up couldn’t give me the feedback I needed.
Here are a few lessons that I learned from my interactions with Jerry. My research partners and I will fondly remember him as one of the most vivid interviewees we’ve ever encountered in our UX adventures.
1. You can indeed gain plenty of valuable feedback from someone who’s extremely high.
I was worried that Jerry wouldn’t be focused enough to provide usable insights. In fact, he told me what I needed to know. But I had to do a little more work to decode it and be a more active listener than usual.
For this project, we were testing a medical website. The site promoted opportunities to participate in clinical trials and encouraged visitors to sign up. Prior participants had told us that they liked the idea of helping others by advancing medical research, but they often did not trust the medical establishment.
Jerry expressed similar reservations, albeit with a far-out twist. He wondered if the government was involved and ambiguously referenced sci-fi movies like Spider-Man as examples of how scientific research could have unintended results for human subjects.
While Jerry’s comparison was unexpected, his concern — that scientists might not know and/or divulge the consequences of their research — was no laughing matter. Trust and credibility are major hurdles that the website had to overcome. So it was meaningful to observe that even a comparatively mellow visitor would crave more reassurance on those issues.
While interviewing Jerry, I had to lean a little harder on techniques that I typically apply to keep research participants on topic and to make sure I understand what they’re saying. If Jerry made a statement that wasn’t clear, I rephrased what I thought he meant and checked with him. I honed my reflection skills and worked harder to make inferences that moved the conversation forward instead of getting mired in confusion.
When Jerry’s mind seemed to wander off the task at hand, I had to always have a question in the back of my mind to keep the session on track.
But, with slightly more insistent prompting, Jerry identified features of the website that didn’t set an obvious expectation for the user (e.g. the link that might take him to another dimension). He mentioned certain aspects of the website messaging that resonated with him. And he helped me and my colleagues notice what points were not salient enough on the website.
2. You do not choose how and when your users interact with the app or website your building. Chances are, they’re not at their most alert.
Because my colleagues and I analyze apps and websites to earn our living, we probably overestimate the level of alertness that users bring to them. Showing up high for a UX session may create unusual test conditions for a medical website, but the situation is actually less artificial than the controlled circumstances of your average scheduled interview.
Sure, your research participant might tell you that your website is a piece of cake to navigate at noon on a Tuesday. But what if it was 11:54 p.m. on a Sunday, when they’ve had a few drinks and realize, “Oh #$*&, I have to pay that bill before midnight!” Suddenly your website or app is going to seem much more like an exasperating obstacle course.
Am I recommending that you include drunk (or high) user testing as part of your research design? Nah, that’s risky and potentially unethical, as Rob Fitzgibbon notes.
Bur remember: a minor inconvenience that a stone-cold-sober user brushes off when they’re giving you their full attention may become a major hassle once the user is tired, distracted, and not being paid to figure it out.
3. When we virtually talk to people in their homes, we’re interrupting their lives. And their lives may bleed into our interviews.
As the pandemic continues to keep people under lockdown, UX researchers will increasingly need to talk to people in their homes. And their homes will be more chaotic on average. Other family members and pets in the background, childcare complications, mounting anxieties, ready-to-snap tempers… all these complications will seep into our research. We should be prepared to deal with all that graciously.
How about when things go back to “normal,” assuming that they do? Remote communication tools have entered the mainstream. So even once the social distancing imperative has diminished, remote meetings will, I suspect, continue to occupy a more prominent place in business and research culture than they have in the past.
In short, talking to people in their homes may be the new normal for UX research. We should expect the unexpected.
4. Our expectations for UX research participant conduct are grounded in our cultural bias as researchers (and as individuals).
Personally, I find that it’s easy to forget that UX research is its own distinct subculture that comes with a whole set of expectations that may seem mysterious and/or odd to others. Researchers can learn a lot by unpacking all that we take for granted about UX testing sessions and how participants should act.
Participants aren’t mind readers. If they haven’t been in a UX testing session before, they don’t necessarily know what to expect or what you the researcher are expecting.
For the sake of convenience, I sometimes recruit through research panels like User Interviews and Respondent. With those platforms, the participants generally bring a shared understanding of UX session behavior norms to the table. Plus, they recognize that their continued income as a panel participant depends on how a researcher rates them.
Panel participants almost invariably join in an eager and alert frame of mind, from a location with a reliable internet connection, dressed like you would for a visit to a bank, in a tidy, well-lit space. Even before the pandemic, they were probably familiar with Zoom and/or GoToMeeting. These participants embrace UX research as a gig. They have a working knowledge of the rules of the game when it comes to UX sessions. But their savvy is a mixed blessing for researchers; familiarity with UX norms tends to correlate with high tech literacy, in my experience. Research panel participants are not always representative of the audience you need to hear from.
For the medical website study in question, my colleagues and I widened our search outside User Interviews’ and Respondent’s research panels. We needed to look beyond the more tech-literate users who were already seeking out UX research opportunities. So, we advertised around the country via Craigslist, then phone-screened participants who fit the demographic profiles we were seeking.
Since some of these interviewees had never participated in remote UX testing before, my colleagues and I spent more time explaining what to expect from sessions than usual. We asked users to join from a quiet place with reliable wi-fi and sent out a detailed participant consent form. Even so, we recognized that the conditions for these interviews would most likely be less controlled than sessions with vetted panel-referred participants.
Jerry is an extreme example, but, after all, I never told him he couldn’t join the meeting while very, very high. His background happened to be the professional culture of nightclub entrepreneurship. Bringing a joint and a cocktail to a meeting? Sparking up a joint on a Zoom call? I wouldn’t know, but maybe that’s just business as usual in his industry.
The unspoken norms of a UX research session, by contrast, reflect mainstream UX culture, which is heavily influenced by tech and corporate culture. We do things a certain way, and it’s arrogant to assume that everybody is familiar with our norms and standards by default. I had become so immersed in those norms that they became invisible to me, but Jerry helped put it all into perspective.
In closing, while I began Jerry’s session with a sense of trepidation, I’m grateful for his input. He was not only generous with his feedback, but he also motivated me to take a harder look at the UX session as a construct.
And hopefully I didn’t harsh his buzz too much.
