Product Managers are Forged in the Crucibles of Launches

Come spend a day in the fire.

I’ve led eight product launches in my career, and whether it was to a user base of zero for Tunezy’s prototype, or to a huge number of active users at FreshBooks, each time has taught me something different.

Launches are unique for product managers in that they demand the breadth of their skills.

Can they tell a compelling story? Ship software that works and is a great experience? Does it fit the company vision or is it a snowflake? Did they do the customer research, or just go on gut? Will they respond to customers well? And most importantly — is what they’re shipping solving the customer’s problem?

The answers to those questions come from the hard work product teams undertake to build something from nothing, and they all come to light on launch day.

PMs live for the launch, and all the excitement, horror, and stress that comes with it.

Are you ready for launch day?


Before Dawn

What should be already done by launch day.


Have empirical evidence that your product solves the user’s problem

I won’t harp on this, but presumably there was a user problem you were trying to solve before you started building. Launch day should not be the first time you find out if it works.

Before even considering a launch, you should have strong empirical evidence (i.e. testing) that your feature is likely to solve the problem for target users. I say likely because test results aren't always conclusive.

In a perfect world, we all have large user bases where we can test new features on millions of users and feel uber confident in the results. Unfortunately, not all products have that sort of scale, and not all features are testable that way.

Which means you better…

Know how to measure success

If you don’t, you’re just gambling. Ensure you know what metrics (engagement, growth, etc.) imply success, and ensure they are instrumented right from the get go. Any product team goes into a launch with a huge backlog of work they never got to — the data that comes in will make prioritizing that list easy. This data should be your obsession in the months to come.

Train people inside the building

When new features launch, calls and emails come quick. If you have support peeps, they better know your feature as well as you do.

Check, check, and check. Can we launch yet?


Safety First.

Launching a feature into a large, active user base brings a whole host of different risks that PMs need to manage. Screwing up can mean serious churn on your product, so take precautions.


Always roll it out incrementally

If you've ever been burned in production before, this is obvious, but for the unscathed — never (ever.) launch to the entire user base, or even >10% of the user base on the first go.

Yes, you tested it. 1,000 times. It’s machine tested. It’s human tested. Your unit tests, integration tests, and regressions are all immaculate. All of them.

I can guarantee that 99% of the time, you've still missed an edge case. 10% of the time, that edge case completely blocks affected users, and ignites a rage cannon.

Humble yourself to the complexity of software being used at scale.

And don’t just roll out to x% → y% → z% (where x < y < z). Take advantage of segmenting your user base to further mitigate risk. Are you a freemium product? Roll it out to free users first. Trial based product? Only to new users, maybe only the ones from Carolina. There’s hundreds of ways to slice it, find the ways that work for your particular product and feature.

Have a kill switch

If the feature can be shut off without butterfly effects, ensure you've invested in the ability to quickly do so (depending on your deployment process, reverting the code might be a reasonable option).

Sometimes though, it’s just not an option. When we launched FreshBooks Payments last year, a feature that dealt with real money from day one, it certainly wasn’t. We knew that once a user collected a payment, we couldn’t pull the plug without serious ramifications.

Even still, we managed the risk where we could. On top of incremental rollout, we invested in two switches: one that would turn the feature off for all users who hadn’t received a payment yet (since they never used the feature), and a second that prevented users who had collected a payment from collecting more of them. That way we could prevent them from making it worse.

Remember, time is critical when launching to an active user base. Being able to shut it down in a matter of minutes vs. hours can mean the difference between 100 broken accounts, and 100,000.

Okay, enough safety — it’s time. 9:58am… 9:59am…


Showtime

The launch itself should run like a symphony.

Hit ‘em up with that sexy launch email. Tweet those tweets you had queued up for days. Drop the bomb that is your Hemingway of a blog post. And of course, update the FAQ.


…oh shit, you forgot the FAQ.

No Launch is Perfect

There is always something that goes wrong, or could have gone better in a launch. This is normal. I’m a believer that if nothing goes wrong, then you’ve probably been too careful and aren’t going fast enough.

The best preparation for launch day shenanigans is a fresh mind and clear calendar. Get a good night’s rest and go for a jog in the morning to prime your brain. Cater all meals for the team, block off your calendar, and lock yourself in the war room for absolute focus.

It’s a celebration

Launches are serious things. They’re the culmination of months of work that could be representing a multi-million dollar bet from your company.

But they’re also celebrations. Of teamwork, of focus, of creativity, of user empathy. It’s a wonderful thing to see your baby learn to crawl. Make sure to take a moment and reflect, high fives all around!

10:01am

Back to work. The first comment gets posted to the blog:

“THIS. IS. AWESOME. You’ve made my life 10x easier!”

You do a little mental strut. Why thank you very much, madam user.

BOOM, they like it. Your team’s hard work was worth it. They are ecstatic.

10:02am

Your team member interrupts your triumph. “Crap we got a bad comment, check out David777."

And they begin to roll in…

“Didn’t work for me. Lame.” — David777
“This doesn’t work for our team, we do [weird use case] and because of that this will never work.” — Cob on the Corn
“Why did you guys do this? You should spend your time fixing [X], I’ve been asked for it for two years!”  Julie McFly

Don’t freak out. Just listen.

When bad things happen, you’ll have to make the choice between activating your kill switch or staying the course. As more hype gets made by marketing, it will get harder and harder to kill the launch without looking stupid to your users.

If you are experiencing any uncertainty about the launch, make sure you tell marketing to stop or throttle the news until you’re sure.

In these situations, your job is to keep your team calm and focus on separating the signal from the noise. Is the user’s issue likely to become widespread? If we continue, can we fix the user’s state eventually or will they be stuck for weeks until we make a major change? Get your team focused on getting the answers to those types of questions, and then be decisive.

This is where PMs need to show their leadership. Many envision the embarrassing debrief with the CEO and it leaves them frozen when they really should just pull the plug. Remember, after “I had to stop the launch, the feature had a bug.” always comes “How many users were affected?”.

Scary outcomes aside, launch feedback is an opportunity. It’s your first chance at full scale research, and there’s a lot you can learn.

Finally, as user feedback pours in, it’s important to respond. If a user is compelled enough to give feedback, you need to ensure they feel heard. It’s the least you can do.

10:30am+

Typically, the rest of your day will be a blur. You’re trying to simultaneously investigate bugs, respond to customers, and orchestrate the next stages of the roll out.

People will constantly be popping in to your room, asking how the launch is going. You’ll put on a smile, reply “pretty good so far”, but in reality you can’t know until much later — most users haven’t even logged in yet.

Lunch comes and goes, the afternoon evaporates, and by the evening there aren't any new issues being surfaced. Your team starts to realize things are settling, and one by one people leave the war room and go back to their desks.

You look over your precious data and send an end of day update to stakeholders, always caveating that “It’s too early to draw conclusions. We’ll learn more in the weeks to come.”

“It’s done”, you think to yourself as you come out of your trance. What now?

Wrap it up with full transparency

Always let the broader company know how the launch really went. Own up to mistakes made, thank your team for their hard work, and most importantly, share the new insights you received from users today and the plans you have to incorporate them into the product.

If you can, let your users know the same. Having users trust that you have their back and are working on their problems is a critical factor in retention. To build that trust, don’t hide behind the e-wall. Open up frank discussions, ask thoughtful questions to your users, and find out what’s not working. Follow up on their concerns, and you’ll earn that trust.

Decompress

Launch days typically end with a myriad of emotions. You realize how exhausted you and your team are. You think about all the hard work, the thousands of decisions made along the way, and marvel at how your team took a germ of an idea, and made it reality.

As you sip a celebratory drink, you can’t help thinking…

See you tomorrow.


Binge reading Product Management?

Here’s a few other posts to satisfy your hunger

The Black Box of Product Management

Why Product Management Exists

Product Partnerships: The Making of a Partnership

Focusing on why and how product partnerships form, part 1 of 2

7 Heuristics for being a Product Director

Lessons learned as a Product Director at FreshBooks

Even more stuff