Research diaries: The one time it’s OK to lie to a user

Justin Ternullo
Jan 31, 2018 · 3 min read

Heads up: This story was originally published in 2015, when Policygenius was much smaller. We’ve grown in both our team and approach since then. :)

Like any startup trying to craft a product that solves a real problem, we’re always user-testing. Whether it’s a completely nascent idea or the development of a new feature, we’re constantly trying to get feedback from our actual users. You could call this an ‘ostrich test’ — referring to the act of removing one’s head from a hole (or anywhere else…) to see what’s really going on. We want to make sure that we’re not building in an echo chamber.

We do a lot of in-person user tests, particularly during the early stages of a project. We invite users, usually with little to no idea what Policygenius actually does, for a one-on-one feedback session. These range from 15 minutes to over an hour.

Where the fibbing starts

It begins right-away with the introduction. I’ll sit down with the user to begin the test, and with a slightly crumpled brow say: “ Hi, I’m Justin — and I DID NOT design what you’re about to see. In fact, I don’t work for Policygenius at all. I’m an independent researcher hired to get your insight on a few ideas they have.” No blinking. No winking. Just a quasi-creepy half smile and my best attempt at smiling with my eyes (see smizing).

Of course none of what I just said was true. As the Chief Design Officer at Policygenius, I did, in fact, design every screen they are about to experience, and I am in no way independent.

So why the lies?

Well, it turns out that the good-natured folks who are generous enough to offer up their time and insights are also too nice to tell me, to my face, that my designs are awful. God love ’em, they just don’t want to crush this designer’s hopes & dreams by looking perplexed at a screen I may have poured hours into. Instead, they’d rather smile back at me and deflect the awkward moment with, “Of course! That makes sense now” or, “I see — that’s a button!” That type of constructive feedback, which usually rears its head in the from of general confusion, inaction, or excessive questions, is better delivered when the well-intentioned user assumes I have no skin in the game.

I call this “host protocol” — or the act of acquiring a completely neutral identity while conducting a user test. The objective is to make the users feel as comfortable as possible, and to give them free reign to bash, critique, question, or generally malign our designs. Policygenius is certainly not the first team to adopt this tactic. Michael Margolis, the UX research partner at Google Ventures, describes the interview process as being centered on making the user feel at home and at no risk of “hurting your feelings.”

From Paul Bloag’s blog (a while back)

And that’s just it — you’d think what’s at risk is having your designs, the interface for the entire site, completely torn apart before your eyes by a pack of insurance-hungry users with an endless-thirst for the perfect insurance-shopping UX.

And you’d be wrong.

The true purpose behind all of our testing, are the little insights, the tiny fireworks that go off in your brain when you see a user stumble on something you assumed was straightforward. The end result is inevitably a string of revelations, on all scales, that make the pages more intuitive and cleanly jerk your head out of the ostrich hole. I’ve never left the room with a frayed nerve, or a [terribly] bruised ego.

Even as our product design team grows, these interviews continue to be the most important part of our week. It’s worth the House of Cards- style maneuvering to see our tweaking and iterating pay off. In fact, take our site for a spin and let me know what you think! Email me your feedback at justin@policygenius.com

Policygenius

The easy way to compare and buy insurance.

Justin Ternullo

Written by

insurance via product via design via dance via magic

Policygenius

The easy way to compare and buy insurance.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade