How we found our biggest leak and reduced it by more than 50% in one week
With Throttle, we want to make sure you get as much value out of the product as possible. For us, that means you understand the product, you understand how to work it, and you use it. And with Throttle, using it = generating addresses.
In pursuit of that goal, we set up a stats page to track how users make it through onboarding and into Throttle. It looks like this:
You see a bar for different steps we’ve identified that a user goes through from receiving access to Throttle, all the way through onboarding, to generating addresses. Each bar represents the number of users that stopped at that step; that did not proceed to any further steps.
Our goal is simple: find the leaks — the tallest bars — try to fix those leaks, and we should see the numbers slowly start to slide further to the right (the last column being the end of the funnel; they’ve made it).
We’ve been doing this for a few iterations now with interesting results, but one iteration in particular I want to show you was the one where we redesigned our previously untouched onboarding process. The results were huge.
Originally, our onboarding went like this:
It was straightforward to us. Install the extension, test it out, pick what time you want to receive your digest, and you’re done. But what we found in our funnel told us we weren’t hitting the mark:
As you can see, we’ve got leaks on the first step (install the extension) and after onboarding — people would finish onboarding but go on to generate no addresses. But why?
We put together some hypotheses, and brainstormed potential fixes.
For losing people at the extension step, we hypothesized that people probably forgot what Throttle was (most signed up in May) and don’t want to install the extension when they don’t know what it is. Our action items to fix: Add a refresher step to the beginning of onboarding with the video, Put the extension step later in the process to engage user to get comfortable before the “big ask” of installing an extension, and Add information on the extension step as to what the extension does (right now, it’s anyone’s guess).
For people completing onboarding, and generating no addresses, we hypothesized that many users may not hit subscribe forms every week, let alone every day. So if we just let nature take its course, it may have been too long since they set up Throttle, and they forget the benefit. So we brainstormed and thought we could change the last step of onboarding to include links to common services’ update email pages, and add a big discovery catalog of curated newsletters below the reading list.
When I went to make those changes, it was clear to me that the entire thing should be redesigned with our newly gathered data in mind. Here’s what I came up with:
Once it was built, we launched it and waited. After three days, we compared the data.
See the final graph, below the original for comparison.
We saw a reduction of more than 50% of people lost on the extension step, and, because it was different, a reduction of more than 50% of people lost on the first step. In closing the second-biggest leak, we succeeded.
We also saw a 46% increase of people that generated one or more addresses after onboarding. In closing the biggest leak, we made a big, big difference, but it’s not totally there yet. The progress was good though: Only 37% of people that finished the old onboarding went on to generate any addresses, but 54% of people that finished the new onboarding went on to generate one or more addresses. For the first time, more people had gotten all the way through our funnel than were stuck at any prior step, which wasn’t true until the redesigned onboarding.
The big downside: because we added 2 extra steps to onboarding, the percentage of people that got stuck during onboarding went up from 29% to 38%. This is what we need to figure out next. It’s all iterative.
Our two biggest leaks are now much further right; they’re the last two steps. Before, our two biggest leaks were the first and last. But now, all the other steps before the last two are below a 10% loss.
So you can see exactly what we were looking for: moving those numbers to the right. This iteration was a great success. But we still have plenty of work to do. The spam won’t stop itself.
