What I Learned at Slack

Part 3: One Metric is Not Enough

I was the first product manager at Slack. I couldn’t be more grateful for all that I learned helping grow the product and the team, so it’s high time I shared a few of those lessons.

Many startups these days are focused on optimizing a single top-level metric. Even more are focused on a single primary data source: analytics, A/B testing, social media, sales, support, internal discussion, etc. Boiling down all of a product’s nuance to one metric makes things simpler. But product development is seldom simple. At Slack, we didn’t take any single data source too seriously. A single metric or data source isn’t enough to truly drive product development.

One key reason is tradeoffs: few improvements in metrics come for free. With every email you send, some will click, but some will unsubscribe. With every feature you add, others become harder to find. With every dollar your customers spend, alongside come higher expectations for your product or service. And worse, a single data source often reflects only one side of this tradeoff. User growth shows up in analytics dashboards, but customer satisfaction doesn’t. It’s dangerously easy to neglect user research when it’s so easy to hear feedback on Twitter.

Some of these tradeoffs are obvious, but others can be obscure or time-delayed. Will focusing on pleasing existing customers ultimately limit your growth into new markets? Could a rush of new users overwhelm customer support and ultimately create a bad reputation for your product? Will fixing bugs as they come in show how responsive you are, or indefinitely delay more meaningful improvements? Questions like these don’t have simple answers. They require a holistic understanding of your business, not one framed by a single metric.

So what’s the solution? Reach for the stars, but set up a safety net.

First, choose clear goals that drive towards a specific vision. Communicate those goals and vision broadly. Clear goals help each team and individual contextualize their work — and find the right metrics to ensure they play their part. Publicly documenting goals at the company and team level lets each employee draw a connection between the company mission and their own work.

Second, set up your safety net. Your vision is a series of bets you’re making on the future. What happens if those bets don’t work out? What’s the flip side of these supposed improvements? Not every decision deserves a detailed cost-benefit analysis, but asking these questions is a good start. Every product decision is a tradeoff. Make sure you’re measuring both sides.

Metrics at Slack

When I joined Slack, the compliments were already pouring in. We’d only just launched, but those early adopters loved us. The numbers were still small, but the growth rates looked promising. So what was I there for?

Looking at the team data, it turned out that many of those compliments were from small technical teams just like ours. Of course they loved it! But everyone knew the future of the company rested on larger teams. So I began visiting members of Slack’s largest teams at the time, with 100+ people.

Those visits opened my eyes. I spoke with people who used Slack every day, but never would write to support or tweet @SlackHQ. Some of them loved Slack, but on average it was more love/hate. I saw what they valued, what they were struggling with, and what they hadn’t noticed at all. We rolled out enhancements to notifications and search that alleviated the worst issues within weeks.

There weren’t many of those large teams at the time, and most weren’t particularly vocal for their size. But knowing how important big teams were to Slack’s future helped us identify a critical tradeoff we had missed. We looked beyond the love and saw that we had ample room for improvement.

Want to hear more about how we used data in product development at Slack? Check out this article + interview in First Round Review and follow What I Learned at Slack for future posts.