User Researchers, It Is Not Your Job To Be Neutral

A common mistake user researchers make is striving for professional neutrality in moderation and interviewing at all times. While this is beneficial in cases where you’re looking to see how a user models a socially acceptable ‘neutral’ response in the context of replying to an interviewer in a professional setting, it is a hindrance to most other research goals.

Don’t be like agent Smith. Staring down your research participants never works, no matter how badly you want insights.

The Goal of User Research

As a researcher your job is not to be neutral; your job is to get the data you need to understand, report, and predict behavior as accurately as possible. To this end, you want your users to be comfortable. You want them to trust you enough to be open and honest with their thoughts and feelings to both you and even themselves. To imbue your participants with this sense of safety, you cannot appear to show a ‘neutral’ poker face devoid of feedback. At best it seems cold and it worst you might cause a panic attack mid-session, all of which leads to a loss of trust and honesty, and ultimately bad data. Participants require a constant feedback loop of acceptance to reveal their true colors with respect to behavior, thought, and opinion.

Case Study

Let’s take Michael. Michael is an approximation of an actual participant I had the pleasure of interviewing as part of a contextual interview study regarding how people do their taxes using TurboTax, the leading software used to complete and file taxes. The goal of our research session that day was to understand what Michael naturally did in product and why he did it.

Michael showed up to the research session well dressed. He had walked through the front facade of the Intuit building, was greeted by Intuits security personnel and politely told to wait in the lobby. When I received him, I greeted him with a “hey there” and a firm handshake. He’d heard of Intuit before and remarked on how it was a “nice place to work”.

Before the session even started, Michael had a preconceived notion of professionalism that was created and reinforced by the pre-session communication sent by the research recruiters, the security staff, Intuits reputation as a tech business, and even the very business-like atmosphere of the campus and buildings. This wasn’t happy hour with the bros, and Michael knew it.

Michael came into the lab with an expectation of formality; he expected this would be a formal affair and that both he and I were expected to be on our best behavior. This was true in respect to how we treated each other; however Michael also took it to mean that he had to show his best side in regards to his behaviors and thoughts around doing taxes using TurboTax, which eventually led to bad data.

Despite explicitly telling Michael he could act and speak as he truly felt and wanted to, he did not fully believe me. He took it as nothing more than a polite gesture. For the first 20 minutes of our session I watched a focused Michael diligently read through all the copy on screen (even the help and marketing copy), triple check the numbers he input, and hold back the urge to guesstimate number he otherwise would have be comfortable fudging.

I began to discern that Michael was not being completely himself by observing signs of tension and heavy pre-processing in his speech and actions. You can peep the guide on how to tell if your participant hasn’t opened up, but let’s get back to Michael.

Michael would:

  • Stop mid-sentence and change what he was saying
  • Take time to prepare a response before responding to a question
  • Imply ideas rather than explicitly state them by using phrases such as “you know…”
  • Talk about the behavior of “some other people”, rather than directly about his own. (EX: “On this screen, some people might not like this color”, etc.)

Once I had noticed Michael wasn’t exactly feeling like he could act natural around me, I began to try and open him up. The goal was to make him feel like he could speak and act as he truly would if he were doing his taxes in his natural environment.

I tried out a few tips and tricks I’d learned before in regards to making user research participants feel comfortable. I found that dropping my own formal ‘neutral’ demeanor, and show him by leading that it was ok to speak and act freely.

If Michael implied he wouldn’t pay a balance due right away by saying “I’ll figure it out…” while looking at the checkout screen, I would explicitly state it back to him as a question such as “Help me understand, would you pay this?” and see if it resonated. If he still acted dodgy, I’d crack a smile and explicitly say “So you wouldn’t pay this?” and assure him that it’d be a perfectly valid answer. In a few minutes Michael completely opened up; His tone and diction had changed, speech became much more free flowing and uninterrupted, and most importantly the content of his actions had changed and his reasoning and descriptions for them was more detailed and personally specific to him.

It turned out that Michael was a cost conscious tax filer with a large family at home. His natural usage was to cut down on time and cost required to file. Initially, Michael read every bit of copy, paid full attention to the numbers he input, and showed what he thought was his best side in interactions with TurboTax. Now felt comfortable displaying behaviors he’d adapted to help him save money while using TurboTax and ignore irrelevant parts of the product.

Getting honest responses from Michael turned out to be crucial, as that study eventually played a role in identifying patterns of behavior for product strategy later on. Had I remained as formal and neutral as I’d been in the beginning of the session, Michael may not have loosened up, and we may have ended up with an inaccurate representation of a detail oriented tax-filer completing taxes the way the current design prescribed. That would have blinded the team to design flaws, and worse, to how our customers truly felt and acted in regards to filing taxes. Remembering the goals of user research to describe and predict user behavior, and the responsibilities of individual researchers to meet those goals rather than perform a role of neutrality is what helped avoid that outcome and led to impactful and actionable results.


You can check out what Boris is up to at www.borisbee.com. We’ve all got something to say, come say it!