Let’s Bring Etiquette to User Testing
Because it’s the right thing to do
I know we’re all busy getting out of the building to test our product ideas, but can we come back inside for a few minutes? We need to talk.
I recently received a Linkedin message from an MBA student.
I saw your post about what it’s like to be a Product Manager. Love it! And would love to talk to you more about it. By any chance, would you have 5 min sometime to give some advice?
I responded “yes!” and gave her times when she could give me a call.
No. She wanted to meet in person. I thought it was weird, then remembered the San Franciscan (and business-student) networking mentality. We chose a coffee shop and she gave me her phone number in case anything changed. Her final Linkedin message before we met said:
Friday sounds great! If it’s ok, my friend would like to join as well. Looking forward to seeing you then!
I arrived ten minutes early and texted her to let her know I was there in a green coat. Ten minutes later, a young woman approached me, tentatively pronouncing my name.
“That’s me, hi!” I said, shaking her hand. We sat down at a table where her friend was already seated across from us, hidden behind a laptop screen. “Again, we just want to tell you how much we enjoyed your article,” the one next to me said, patting me on the shoulder. I nodded graciously.
“So based on your message I assume you are both considering product management?” I asked. The friend answered with a question.
“Why don’t you tell us about your college job hunt. Did you always know you wanted to be a product manager?”
“Well…Um. No.”
“Why? What was your dream job?”
“Um—”
“And did you even know what product management was?”
This is so weird. I thought they wanted advice, I wondered. The rapid-fire questioning continued.
“What other jobs were you considering?” “Do you think that if someone had given you one-on-one guidance, the search would have been easier? What services did you use, anyway?”
Fifteen minutes later, I realized what was going on. Oh my god. They’re doing secret user research. My heart beat faster. Should I say it? My armpits moistened. My cheeks flushed. “Are you guys doing a research project?” I resisted the urge. I’m sure they’ll clue me in at the end.
Still hiding behind her laptop screen, the friend remained detached. This didn’t feel like human interaction. As the interrogation continued, I was less inclined to provide detailed answers.
“So what if you could have gone to the library and watched videos about different jobs? Would that have helped the search?”
“No.”
They were violating me. I hadn’t provided consent for this research, whatever it was.
We hit the thirty-minute mark.
“I know you have social plans. I don’t want to keep you from them.” The first woman said. Relieved to stand up, I skipped the handshake and turned to leave.
“By the way, keep writing,” the friend offered. “Just don’t stop writing.”
It was the grand finale of amateur hour. I walked out of the coffee shop, somewhere between upset and anxious. Are they who they said they were? Well, she sent me a Linkedin message…
I was halfway down the block when I got a response to my “I’m here and in a green coat” text message. The response was a question mark. Nothing else. Seriously?! She gave me the wrong number. I looked to my right and left.
And then I ran.
In July 1961, Stanley Milgram conducted one of history’s most controversial psychology experiments. It was controversial because the experimenter deceived participants and the deception caused participants to feel extreme emotional distress.
The controversy spurred industry-wide attention on what was appropriate when doing lab research on humans. Today all studies are screened for their compliance with a specific code of ethics.
Effective for more than 50 years, this code is revered by psychological researchers around the world.
Getting product ideas in front of customers — interviews, paper prototypes, dry landing pages — is a necessary first step before we build anything. But especially when we’re not in lab settings, our tests deceive users all the time.
Dry landing pages encourage people to sign up for products or services that don’t actually exist (yet). Concierge MVPs lead customers to think they have purchased a well-oiled product when in reality, it’s a scrappy prototype. Behavioral interviews are conducted under the pretense of general market research.
Methods of lean testing make it easy to forget that we’re dealing with real people. Though extreme, my coffee date was a perfect example of this problem:
User testing is missing a code of etiquette.
The code can’t be as cut-and-dried as in psychological lab research, but we can maintain some respect in tech. Etiquette is as important in testing as it is in everyday social interactions.
We can validate our product ideas without being rude, pushy, and deceptive.
Here’s how:
1. Provide context
User testing is not a surprise party. Ambiguity will stress people out and negatively impact the quality of their responses.
When in person, introduce yourself and explain what you’re testing. It can be vague, but anything is better than nothing.
“I’m a product manager researching this specific problem that pertains to you. Can I get your thoughts on an idea we’re working on?”
If you’re looking for a natural behavioral response and giving away the context would taint that response, provide context at the end.
Online, be clear about the product or service you are offering.
2. Explain the tester’s purpose and rights
Participant bias occurs when people try to act in ways they think will please the experimenter. Prevent this bias by telling users you’re looking for their brutal honesty.
“Please give us your most honest feedback. If you find something to be confusing, chances are other people find it confusing as well.”
If your tester becomes uncomfortable at any time, back off! There are plenty of willing testers out there.
“If there are any questions you’d rather not answer, that’s completely fine.”
3. Ensure confidentiality
In person or online, make it clear if and how you will use personal information.
“We won’t share your information with anyone” or “We hate spam too. We’ll only email you with important updates about this idea.”
4. Express gratitude
People have a lot going on, and you should feel lucky when they go out of their way to help your product or idea succeed. It’s only fair that you take a moment to thank them.
This can be monetary compensation (Amazon e-cards work well), education (“We’re working on solving this problem. In the mean while, did you know…”), or even warm and fuzzies (“Thank you so much for your time. This has been so helpful for us in understanding…”).
5. Debrief
Once you’ve observed the behavior or attitude that you are testing, notify your participant that he was part of an experiment and allow him to ask questions.
This doesn’t necessarily need to be long or intimate. Online, think dialog box.
Why does this stuff even matter?
Setting aside basic courtesy, it matters because we live in a small, connected world. With each prospective user of your product, you have one shot to make a good first-impression and testing is part of that. If it doesn’t come with serious legal implications, disrespect and deception will leave a bad taste in people’s mouths. This, in turn, will hurt you.
It has been two weeks since my unsettling coffee, but I am still on edge. It took five days for me to receive closure, which came in the form of a one-line Linkedin message thanking me for taking “the time to chat.” If and when the young women’s business-school project hits the proverbial shelves, I won’t promote it. That’s how icky they made me feel.
Don’t let this happen to you. Remember, your testers are evaluating you and your product idea. The reputation of your future product is up to them.
