by Sam Richard, Director of Growth at OpenView
What do the SATs, Body Mass Index Calculators, and OpenView’s SaaS benchmarks have in common? They let you know where you stand amongst your peers. As someone who works with a number of different software companies, I’m a big fan of benchmarking and was pretty excited when two companies that help enable product led growth published their own product benchmarking reports with their own users’ data, so product-minded operators can see where they stand.
Mixpanel and Pendo are the two vendors that released benchmark studies this year, which is great news for product practitioners everywhere. Some of the key takeaways I found from the reports were:
- Actual product retention (usage of the product on a regular basis) has become just as important and closely measured as revenue retention (cue the applause from me!).
- The whole market seems to be looking for a singular metric to help product owners understand product engagement with one single number, at a glance. Since engagement can differ based on the number of features your software has, or its intended use, finding that one-size-fits all metric is really hard, and both Pendo and Mixpanel have been incredibly innovative in working to create it.
- I realized there’s still so much room in SaaS to build for the end user. Perhaps it was because Mixpanel broke out SaaS metrics from other industries, or because Pendo shared the “average” NPS (25) for their users’ products. B2B SaaS users are people too, and they want the software that they use at work to treat them that way.
Pendo
First off, I was so impressed by the way Pendo visualized their data in their benchmarks report, it’s polished and shows that they’re really committed to providing guidance for product professionals. Kudos again to Pendo for breaking datasets down into quartiles so you can see where you stack up, not just the average.
Stickiness and adoption
In terms of the actual metrics, Pendo led with a measure of “stickiness” by taking DAU/MAUs and WAU/MAUs. I tend to get a little nervous when I hear businesses talk about counts of users showing up in the application with regularity to illustrate engagement, because there are some outlier products, like Expensify, where I only get value out of them monthly (I’m sure there are people who do expense reports weekly, or daily, but I’m glad I don’t belong to that minority).
Pendo works to isolate engagement further by indexing feature adoption, by counting the number of features that generate 80% of click volume. Indexes are great because they create a new metric to measure against, one that should be common across all companies. My chief concern with an index that takes into account the number of features users touch puts simple, repeatable SaaS products like Calendly at a disadvantage. Do I use many features in Calendly? No, I integrated it once, and I use that 30-minute meeting link daily, does that make me less engaged?
Using these two touchpoints, Pendo created their own product engagement score. This score is valuable because it allows product owners to know at a glance how they’re performing. On the other hand, like any one-size-fits-all metric, it suffers from not providing coverage for a number of B2B SaaS products.
Retention
In the Pendo study, they’re talking about usage-based retention, or where the user keeps coming back to the product over and over again. What’s shocking is that even for best-in-class Pendo subscribers, 44% of their users leave after a month. With rising customer acquisition costs, increased competition, and no-code or low-code solutions enabling faster feature creation, software companies have it rough. If only 66% of those users you finally manage to get inside of your product actually stick around in the BEST case scenario, you need to figure out how to provide value immediately. Sound familiar? All of these factors are why product led growth (PLG) is becoming the go-to-market motion of choice for fast-growing SaaS businesses.
NPS
Finally, Pendo benchmarks Net Promoter Score (NPS), as their product collects it in SaaS applications. While I love the idea of measuring end-users’ product experience, I have strong feelings about NPS, and its quality as a metric. We can do better, can’t we?
Mixpanel
Mixpanel has a bit more diverse user base than Pendo. Mixpanel has gone deep on providing product analytics, while Pendo has other features for product owners, like NPS collection and in-app guide creation. While Mixpanel has many types of businesses (not just software) on the platform, they did separate out SaaS companies in their analysis, which is what I used for comparison. Structurally, I love the key takeaways at the beginning of the document, especially their finding that the 90th percentile of businesses using Mixpanel are doing 18x better than the median on the metrics they selected. Now, onto those metrics.
User Growth
These numbers were interesting (3–13% growth in new users) because they could be extremely variable based on a business’ go-to-market strategy. One cut of the data I’d love to see is user growth in freemium vs. non freemium models. One surprising fact from the data is that people are now using mobile for work, lessons to be learned here from Slack.
Activation
It’s nice to see activation being used broadly! Even a year ago, activation wasn’t on many people’s radars. However, do the actions that Mixpanel selected: Subscribe, View report, Create account, Complete application, Download and Check out truly provide value to the end user? This could lead to enormous variability across products to identify true activation, as some products don’t have any of these actions. One trend I’m seeing and would love to see more of in SaaS tools is allowing users to create their own version of activation mad-libs style with their data. Perhaps with this rise, we’ll start better understanding activation in the next few benchmark studies that come to market.
Active Usage
I have to return to my prior observation from the Pendo benchmarks: For some Saas products, daily and weekly use isn’t always a sign of user happiness. Every product has a different use case. Product managers, be wary of benchmarking your own products for this particular metric.
Engagement
It seems like everyone is creating their own engagement score these days, and I love that. With the Mixpanel report in particular, I’d flip the high number of key actions per week on its head to argue that there’s a ton of room in SaaS to better design for the end user, and streamline actions so users aren’t clicking around as much.
Retention
I love seeing this again! My favorite quote from the whole benchmarking study was, “As the cost of user acquisition increases every year and in every industry, customer loyalty becomes the primary path to profitability.” This sounds like PLG to me! I will say however, I was very upset that SaaS retention benchmarks weren’t reported on.
Takeaways
Product owners are finally able to look under the hood of other products out there and benchmark their own performance. That’s a big deal! Both Pendo and Mixpanel are performing an amazing service for the product community. I can only imagine how much more valuable meetings I’ve had in the past would have been if I’d had access to these reports back then.
On the other hand, I’m concerned that the number of metrics a PM is now expected to manage is fairly unlimited, and a few of those metrics haven’t evolved with the SaaS landscape. The product-led growth collective has laid out 6 metrics they feel are particularly valuable for PLG PMs, but it’s hard to understand where you truly stack up against other PLG companies with the benchmark reports out there today. We’re working on something special here at OpenView, and if you’d like to contribute and find out how your own PLG benchmarks stack up before anyone else does, sign up here.
