Sell More Coke: Teaching a Quantitative Organization to Love Qualitative Data

Maureen Be

Yammer is a data-driven company, which means we use analytics to build, test, and iterate on every feature to make sure what we’re doing has a positive impact on our users. When I joined Yammer, our Product and Engineering orgs were very skeptical of qualitative research. Compared to knowing stats of thousands and thousands of users, how could insights from a five person study be useful? (Side note: here’s why)

One afternoon, frustrated after presenting a poorly-received overview of past months’ worth of qualitative research, manager Cindy Alvarez took the team out on a drive. At that time, Research was fairly new and we were in the middle of fighting an uphill battle to get the rest of Yammer to see the value of qualitative research. After driving a while and passing lots of billboards, she broke out into her metaphor-speak (anyone who has worked with Cindy knows she does this often):

“It’s like the basic concept of advertising. If you keep seeing a Coke ad over and over, you’re going to keep thinking of the Coke and eventually you’ll buy the Coke.

I need you guys to sell more Coke.”

From that day forward “Sell More Coke” became our team’s official-unofficial mantra, and I’m happy to say that our Product team is now happily drinking that delicious, refreshing beverage (this is an unpaid message not brought to you by Coca-Cola). So how’d we do it?

Know your target market, and speak their language
Be educated in how you’re speaking to your audience. Know who you’re talking to, where they’re coming from, and understand how they’ve been getting information. Since Yammer likes numbers, we present qual data alongside quant data as much as possible, and we make sure we’re just as good at asking analytics questions as research questions. Make friends with data analysts, and learn some stats concepts and how to analyze A/B tests. Learn how to contribute to conversations outside of (but highly related to) your area of expertise.

Be assertive, eavesdrop, and interrupt
Don’t be afraid to stir the pot — no one’s going to care about you until you get their attention and tell them why they should. We caught and challenged assumptions when we heard them and interjected ourselves into countless conversations. We jumped into everyone’s meetings, and explained why getting research involved earlier in the development process could not only help us move faster, but get to the right answers quicker.

Work quickly
Show everyone how painless it can be to get feedback. People had a perception that research would take weeks or months. Instead, we turned around research requests quickly — and I mean very quickly. We shared survey and click test results within a day and interview synthesis within a week.

Get people involved
Make it their research as much as yours. Our engineering org had never been exposed to our customers, so it was easy for them to think of our users as numbers. To ground everyone, we conducted rounds of interviews and volunteered (read: bribed with the help of candy and booze) everyone on the project team — developers, QA, PMs, designers — to sit in on at least two interviews and participate in brainstorming sessions. All it took was hearing the voice of a real user to turn those empathy light bulbs on, and we started hearing people advocating for our users.

Put those ads up everywhere
Get used to saying the same thing over and over — really beat it in. Especially when you feel something is common knowledge, it’s most likely not. You sell that Coke. Sell it.

So what’s it like at Yammer today? Now, at the beginning of each project, we go through an ideation phase where we share relevant analytics AND research, and both teams work together to identify areas of research that will help from the get-go. Along with quick, reactive projects, we’re working on bigger-picture areas of research that’ll help frame the future of Yammer and Microsoft together. And it tastes so sweet.

Maureen Be is a User Researcher at Yammer. You can tell where the cutest dog in the office is based solely on the pitch and frequency of her delighted squeals.