Vanity Metrics vs. Core Metrics
Metrics are key for running any kind of business, especially early stage startups. You need them to show progress to yourself, to investors, to customers. You need them to decide what to build, who to sell to, and where to invest resources.
In each industry people have different metrics they cherish. It can be number of downloads, number of transactions, active users, signups, time to service, customer satisfaction (NPS), MRR, ARR and many others.
Each of those metrics is important and relevant for specific aspects of your business. But none of it can cover all aspects at once. There is no one metric that rules them all. Here’s how to know what’s important and what’s just a squirrel you shouldn’t be chasing.
So What Are Vanity Metrics?
Vanity metrics are often “big numbers”. The stats you read in tech magazines and press releases, the shock and awe of the numbers game. They are what you use to impress investors, partners and essentially anyone who asks to hear your pitch.
If those numbers are indeed helpful and important, why do we call them “vanity”?
The best example I know is with the notorious metric: number of downloads.
While downloads can definitely indicate public interest and signs of growth, they tell you nothing about the health of your business and the quality of what you’re selling. Take a look at your phone: how many times have you downloaded an app that you only opened once?
While being helpful in some respects, relying on those metrics alone is highly dangerous and can often drive you and your company in the wrong direction. What’s better: one customer who uses your product 10 times or 10 customers who use it once and leave? We’re looking for that one customer and then trying to figure out what converted them to an actual user.
The more value your customers get, the healthier your business is.
Fortunately, there are metrics that represent actions or behaviors that drive value for both you and your customers. The more value your customers get, the healthier your business is. These core behaviours are very different between industries and types of businesses. Examples include: a service company may use the percentage of returning customers and user reviews or a Saas company may look at the amount of time spent daily in their app or website.
The important point is that those numbers (i.e. daily time spent) will give you clear and objective indications that your customers are getting real value and are happy to be your customer.
Those are your core metrics.
Finding My Core Metrics
For a while, the metrics I cared most about were number of signups and retention over time.
Those are definitely important metrics, but I came to realize they did little to help me make the right decisions for each customer.
As you probably know, in Saas retention is everything. But measuring retention is a tricky business. Take this cohort analysis chart as an example:
You can easily see that users for that company’s website or app are mostly gone after the first day and completely gone after the 5th day. If number of days is a core metric for this company, that’s a bad state to be in.
But while cohort analysis gives you a good status of high level retention, it’s a metric only in retrospect. Why? You only see users once they’re gone and by then it’s probably too late. Now you’re paying twice the cost: once to acquire them and again to replace them.
My cohort chart looked good, and looking at it every week was a good high level pulse. But it did nothing to actually help me improve my retention rate or at least understand why people leave. Or the real question I needed to ask: what makes them stay? If you can figure that out, you’ve found something that can drive real growth. Lather, rinse and repeat (well, mostly).
What IS a Good Core Metric Then?
It starts with a gut feeling. I knew what feels like a “good” customer to me, I just wasn’t sure how to measure it.
I was looking at multiple different factors. For example: Specific title of the first user, size of team, industry, type of content they enter to our system etc. All those are important properties but were not a consistent indicator of success nor were they my core metrics.
For a metric to be a good core metric you need it to represent a behavior common to all the cases where it “felt” like a customer is going to be successful. Instinct is good, but you’ve gotta back that up with data. You need metrics that can be translated to operational or functional changes, so you can track your progress and improvement over time.
If it’s not actionable and can’t drive decisions, it’s not a core metric.
Finding Where The Signs Lead
After many hours of manually crunching raw analytics data and talking to our users, I figured out what makes a customer “good” for our company.
Teams that use our product are varied in size and shape, and can grow over time to include large number of users. But my core metric turned out to be the engagement of just one single type of user: the ‘Manager’.
When the Manager is happy — when they actively use the system every day — the entire team follows.
Tracking my top customers’ behaviour around the Manager’s engagement allows me to be proactive about retention and help my team take action to help those customers before it’s too late.
To keep an eye on that behaviour, I’ve pulled in to the same view all the different indicators from Google Analytics, Mixpanel, Hubspot and our own SQL, including the action items relevant to this account (tasks and bugs from Jira). This is something that everyone can easily do with Tonkean. That data is automatically being tracked and Tonkean alerts me and the account manager when the metrics change drastically, so we can take immediate action.
But metrics alone are not enough to be preventative. They’re more like a smoke alarm, alerting you that something needs to be done. Many times a human touch is required. Knowing the status is important but knowing what to do about it is the real key. If you hear the alarm and ignore it because it doesn’t mean anything to you, there might as well be no alarm.
The context that my team provides in Tonkean about their latest contact with the customer, such as user excitement levels, the fact that user is currently traveling or any other information that can shed important light on the metrics, are critical to making relevant decisions. If you see a dip in the Manager’s numbers but know he’s away on vacation, you know you don’t need to react. On the flip side, if that same metric dips with no apparent cause, it’s time to step in. Having a place in Tonkean where both the relevant data and the user context can be joined is what made all the difference.
Wrapping It Up
Finding the right metrics is key for any manager and a startup CEO in particular. Your core metrics must represent the value your customer are getting from your product or service and must be quantified enough to drive decisions and actions.
But most importantly, make sure you have the operational structure that gives you the right context to really understand those numbers at the right time. Otherwise, you’re chasing a squirrel and leaving your customers behind.
Thanks for reading! I would love to hear your own stories, feedback or thoughts. Please feel free to comment here or reach out via email or Twitter.
To learn more about Tonkean, check out: https://tonkean.com
