Users are collaborators, not test subjects

There’s a lot of product literature out there, but the one our product managers’ book club chose to kick off with is Product Leadership: How Top Product Managers Launch Awesome Products and Build Successful Teams.

One word unsurprisingly came up a lot: users.

We’re so often reminded: “The product manager is the voice of the customer inside the business, and thus must be passionate about customers and the specific problems they’re trying to solve.”

Mic’s product team had our first experience with in-person user research in the spring. We conducted this research in an attempt to answer the following questions: How do users assess the credibility of a news outlet? What features help users make this assessment?

The tricky part about this research is that it does not lend itself well to quantitative analysis. Assessing the credibility of a news outlet is an intellectual activity; it would be hard to measure this assessment through things like click events. Engagement metrics wouldn’t really tell us much about a user’s general view of Mic as a news source.

As Product Leadership underscored: “Getting the balance right between qualitative and quantitative, it turns out, is the key to unlocking true customer insight.”

A great qualitative research mechanism is real-life conversations. User interviews allow product workers to see how actual users think through features and flows.

So in late May, we invited 19 users to our office to evaluate 13 feature mockups. The interviewers spanned departments, from video to analytics to product. For everyone at Mic, it was a humbling experience. We were — each one of us — surprised by users’ responses to the features we demoed.

After reviewing the comments we heard from these sessions, we identified several features that elicited consistently positive reactions. We plan to ship these over the course of the next few months in collaboration with our newsroom.

This was the first time Mic conducted user research for product development purposes, and we learned a lot! We appreciated the opportunity to interact with real users who we would normally speak of abstractly and with many assumptions.

Looking ahead, it is our goal to proactively make user testing a routine activity. We would like to explore starting the process much earlier, even before mockups or prototypes exist. We want to get to the crux of our users’ problems: Why do people consume news in the first place?

As Product Leadership noted, “Asking your customers which UI elements need to be changed will get you only a fraction of the information you need to make a product better. Discovering the real reasons customers use your product will leapfrog the basic interaction questions and go straight to the core of why they think it’s valuable.”

How have you conducted successful user research? What are some good tips you’ve found for effective user research?

Let us know in the replies!