What billions of user events taught us about SaaS products
Disclosure: Mixpanel, the Data Analytics company, has previously sponsored Hacker Noon.
Everybody wants to know how their metrics stack up — and because Mixpanel is a data analytics company, we have the unique ability to actually deliver some answers for companies and product teams that have questions about their own user behavior metrics.
To get those answers, we had to put together a comprehensive dataset. So, we aggregated behavior data from 1.3 billion unique users who triggered more than 50 billion web and mobile events across hundreds of products run by our customers. The goal was to give product managers, developers, marketers, analysts, and other folks on the product team a clearer sense of how to measure themselves in key areas of product health: usage and user growth, engagement, retention, and conversion. By focusing on these four areas, we aimed to show what happens across the full user journey.
We then bucketed the products surveyed into four sectors: software-as-a-service (SaaS), e-commerce, financial services and media & entertainment. Among those industries, SaaS stood out in one key way: its business model. SaaS is weighted heavily toward the business-to-business (B2B) model while the other sectors are either exclusively business-to-consumer (B2C) or some combination of both. SaaS also just so happens to be the industry Mixpanel is in, so we were particularly curious about the insights we found here.
In this article, we take you through some (some!) of what we found in SaaS.
This is story is brought to you by Hacker Noon’s weekly sponsor, Mixpanel. For deeper insights, download the full report.
Usage
Usage is the difference between relevance and irrelevance. Metrics like stickiness and ADAU growth can help you understand the beginning stages of the user lifecycle for products.
First up is stickiness, which tries to show how active your use base is. To measure stickiness, we looked at DAU/MAU, which measures daily active users divided by monthly active users. For the purposes of our dataset, someone counted as a “daily active user” if they recorded an action of any kind in an app in a given day.
This graph shows average DAU over the course of a month divided by MAU for that month, expressed as a percentage. We looked at average and median (i.e. the 50th percentile) to understand what is “normal” for the industry, and then we found 90th percentile stickiness, to show what best-in-class performance looks like.
As you can see, there is a major gap between the 50th percentile and 90th percentile in stickiness of almost 20 percentage points! In fact, of all the industries we looked at, SaaS had the highest median stickiness, and even then, that median 9.4% number implies fewer than three days of activity per user per month.
At the top of the spectrum, stickiness of 28.7% is the gold standard for SaaS products. This translates into more than eight days of activity per user per month, or approximately two days of activity a week. If people are using your SaaS product a couple of times a week, your product is performing at an elite level.
We also looked at growth rates in SaaS. By ADAU, we mean average daily active users, with active users defined as active if they performed any action of any kind within the measured day. ADAU is the average of daily active users over the course of a month. For this graph, we tracked month-over-month change in ADAU.
While ADAU growth is not the only usage metric that matters, in general, flat or negative growth is not a great sign. Not the deepest insight, but it’s true: it is exceedingly difficult for a product to generate value without user growth. SaaS in particular looks pretty brutal — the 50th percentile sees negative ADAU growth, showing just how competitive it is out there. On the other hand, the top performers are growing quite quickly at 17.3% per month.
While growth goals should vary depending on a company’s maturity, consistent growth adds up. A company that can maintain the 90th percentile number of 17% ADAU growth month-over-month will have doubled its number of active users within five months.
Once again, the delta between what it means to be at the 90th percentile in ADAU growth and the median is pretty staggering. Shameless plug: product managers can use Mixpanel to track their ADAU metrics and get a better sense of what kinds of actions lead them closer to the 90th percentile.
Retention
Retention in this case means: did a person perform an action — any action — and then come back and perform another action again. For median SaaS companies, that means 37% come back one week later, a number which will dwindle to only 15% by the end of eight weeks. By comparison, elite products still have retained over a third of their new and existing users after the same period of time.
Retention has a lot of factors from the most obvious, like whether or not users find value in a product, to subtler ones, like which platform users access a product on.
For most SaaS companies, retention does not necessarily equal satisfaction with the product. If a customer has a paid subscription, they are likely to use it whether they like it or not. You’ll get a clear picture of whether or not they’re satisfied when it comes time for them to renew their subscription.
Engagement
For our report, engagement is the sum total of all actions performed in a given product. This graph is normalized around the average day of the week mobile and web engagement. That means that a day that measured as “0” on this graph would be one where engagement was equal to that of an average day. A day measuring at “15” would have 115% of the daily average for engagement, and one measuring “-15” would have 85% of the daily average for engagement. That way, we could compare companies of different sizes easily.
If you have a SaaS product and you’re trying to increase engagement on the weekend, good luck to you because overall engagement plummets. The bottom line: these products are generally apps that people use for work, and the weekends are not for work.
On the web engagement bars, you can almost see the life force seeping out of employees as the work week progresses. Just look at them, all chipper on Monday, using the app 18% more than average, and then steadily engaging less and less as the week slips away. Interestingly, both web and mobile usage follows the same pattern of engagement.
As such, product managers should consider launching features on Monday or Tuesday when overall engagement peaks.
Conversion
In the case of SaaS, we defined conversion as converting from an account signup to actually subscribing.
Holy smokes this number is low! 3%! And unlike with other sectors, where we started measuring conversions from “view report,” we started from signup because otherwise the rate would be even lower than this.
Because SaaS is asking businesses to subscribe to a service, its conversion numbers are unsurprisingly lower than products in other industries. The benefits of this approach are clear, though: once a customer has bought a subscription, all they have to worry about is renewal.
Putting it all together
The median SaaS company has users that visit 2.8 times per month, though its count of active users is decreasing by 5% month over month. In fact, after a week, only 37% of users return to perform any action at all, and after two months, that number is 15%. Median SaaS traffic comes largely during the work week, though it does decline from Monday through Friday. And for all of this, it is only able to convert 3% of those who sign up into actually subscribing.
In short: it seems really hard to be a median SaaS company. Download the full 2017 Mixpanel Product Benchmarks Report to see more data on SaaS products, so that you can build better products and move closer to the 90th percentile in the metrics that matter most to your company.
