#UXRConf Preview: Meet Gregg Bernstein
#ama with Gregg Bernstein, Sr. Director of User Researcher at Vox Media on removing bias and barriers
Before the 2019 UX Research Conference in June, UXRC will be a hosting a series of Ask Me Anything conversations (#AMA) with some of our Conference speakers! 🔥
Our first AMA was with Noam Segal, Director of User Research at Wealthfront. And today we hosted our second conversation through our Slack community with the Sr. Director of User Research at Vox Media, Gregg Bernstein! 👍
Gregg Bernstein establishes user research practices for growing organizations. As senior director of user research for Vox Media, he spends his days advocating for and practicing user-centered product development. He speaks around the world about his work at conferences, most recently at Webdagene 2018 (Oslo), UX Scotland 2018, and UX Insight 2018 (Utrecht), and for such companies as Sprout Social, Booking.com, CA Technologies, and more.
On June 6th, Gregg’s talk in the Leadership in Research track will be called: You’re already a researcher. It will be about the importance of sharing information across your organization in order to create a common goal and great user experiences. 🙌
Here’s what Gregg had to say.
What are some common rebuttals/strategies you use to address stakeholders who voice concern about small sample size? — Taylor Kim
For context, I’m typically working with teams who have done little to no research. Often my goal is to remove the barriers to further research by getting people comfortable with the idea of even running an interview or a test, or sending a survey.
So my rebuttal is usually “sure, we only heard from 100 people, or we only tested with 5 users… that’s more than the 0 we heard from previously.” Even if all respondents or participants are edge cases, it’s still rich data — we’re learning about actual users and uncovering information we previously didn’t have at all.
In working with folks from other disciplines, how do you help your collaborators avoid bias in their interpretation of data? — Carrie Heffner
First, I try to be as upfront as possible about opportunities for bias. We did a study of readers of our site Polygon.com recently, and the opening of my analysis says:
- Note that demographic data might differ here from what we see in [other data sources]. This survey data is based on responses from folks motivated to offer feedback to Polygon, and might not encompass a sampling of all possible consumers.
So that’s me hedging a bit and acknowledging that we’re already working from a somewhat biased group of people.
But another way I think about this is:
- This research is meant to be applied. It’s for a product.
Sure, the sales team might be biased in their interpretation, but that’s because the lens through which they view research is “How does this help us sell more_____?” Same with my product team — we need to balance what the data tells us vs. where media consumption is headed, and that’s a bias that should be reflected in our analysis. Which I usually call out as “This is my perspective…”
I try to be as upfront as possible about opportunities for bias.
What are the best strategies you’ve seen for involving a wide variety of stakeholders in research internally? I’m thinking folks like sales teams and engineers, but I’m curious to hear what types of people at Vox and MailChimp have gotten involved in the research process and how you’ve brought them along and/or incorporated their perspectives? — Dan Epstein
I have some guiding principles I follow in my work, two of which are:
- Be more certain
- Make everyone smarter
I want everyone to make better decisions, which means giving them info I have that can reduce risk or allow them to plan with more certainty. So at Vox that means finding the people who would probably find the results of my study useful. Sales and marketing teams always want more information. Support teams always benefit from additional user information. So I find an ally on those teams and start an informal data sharing relationship. “Hey, here’s some stuff I found this week — is this useful to you?”
The best part of this is often these teams have info that will help you too. I now see sales dashboards, or hear about upcoming initiatives that might impact my own work. I’ve now established enough of these relationships that research is a known entity across the org — those transactions and informal knowledge shares grew into more formal relationships, and now my mandate is to be the advocate/interface between our org and our audiences/users. Researchers are diplomats, forming relationships and finding common ground. Those relationships are so crucial to research success in an org.
I have some guiding principles I follow in my work, two of which are: Be more certain and make everyone smarter.
I’d love to learn more about how you scaled research practices in a growing org? How big was the research team when you got involved? What is your advice for another company who is growing their research team and the practice as a whole? — Dilan Ustek
At Vox I was the first user researcher — before I arrived research was practiced by designers and PMs. I was hired to embed on a specific project, but I made it my business to show the value of researching everything. It’s all one user experience — users don’t care about different business lines or products. It’s one thing. So my personal mandate was to prove the value of researching the holistic user experience of media consumption. I carved out time to do studies of how people find and subscribe to podcasts, newsletters, or Youtube videos. I studied what a Verge reader looked like.
Once people started counting on this type of work, there was demand for it. No one wanted to go back to *not* having this research. So the scaling of research was very intentional; if you want to grow a team or practice, the best path is to make it where folks couldn’t imagine an alternative scenario.
It’s all one user experience — users don’t care about different business lines or products.
Can you speak to a time in the last few years where you really missed the boat on a research project and what you learned from it? — Alec
I once discounted a small sample (people complaining about a design change); only later did we realize the small sample was a group of people who no longer found our product accessible. That was informative for me — I realized that small samples are huge opportunities to learn from a group of users.
At MailChimp we learned that one of our competitors at the time had worked with the Small Business Administration and local Chambers of Commerce to get their product recommended to new business owners. It was a real smart strategy, and I wrote up a proposal to learn more about this/see how we might do something similar, and I never fought for it. That is a regret I have — as the research leader, it’s my job to communicate my viewpoint, not just write it up. I learned from that, for sure.
For researchers newer to the UX field, what is the number one piece of advice you would give your younger self in terms of building your UX research skills and practice? — Carla Williamson
All of us have research skills. We shop for phones or cars; we make complicated travel plans. We all have the ability to find and synthesize disparate bits of data to make decisions.
My advice is — whether it’s your job or not — be the person who connects the dots in your org. Don’t ask — just go do it. Find the people who have useful info, make friends, and then share what you learn. No one is going to say, “Ugh, why are you doing the hard work of helping us make better decisions?!” Being the dot connector is a good way to build skills in finding and communicating research for different people in an org.
Join Gregg Bernstein at Strive: The 2019 UX Research Conference
Purchase tickets here
📅 June 6–7
📍 Roy Thomson Hall, 60 Simcoe St, Toronto, ON, M5J 2H5
