What makes a good metric?
Last time we did a short introduction on analytics. We explained that numbers that can only go up and to the right are called vanity metrics. Metrics that only exist make you feel good. For example, the total amount of signups since the beginning of your startup is a vanity metric. In this blogpost we will dive further into what criteria make good metrics.
Understandable
Metrics need to be understandable. If you cannot share your metrics with the rest of your team without having to explain them first, they are too complex. Your team will forget about them and will not use them in there day-to-day activities. Make them as easy as possible to understand, easy to communicate and try not to track too many of them.
When I first started out with Google Analytics, I tracked every page and event that I could think of. I never looked at that dashboard, because it gave me no insight. I could not see the forest for the trees because there was too much noise. Nowadays, when I implement Mixpanel I begin with tracking the main activity of a product and then add more only when needed. I try not to track more than 5 or 6 metrics at a time and always focus on only one.
Comparable
If you can not compare the metrics with data in the past, across cohorts or for example across clients, you can not run your business on it. You always need to be able to compare your metrics. Only then do they support you in uncovering the real problems in your startup. Otherwise they only make you feel bad. (Or worse, they make you feel good)
Vanity metrics are for example not comparable. 15.000 downloads is neither good nor bad, while 1500 downloads today sounds pretty decent!
Behavior changing
If a metric does not change how you behave… It is not a good metric.
This final criteria of a good metric is by far the most important one. As long as a metric does not change your behaviour, it is useless, since it does not help you reach your business goals. How will your metrics change the way you run your business?
1.3M total signups do not change your behaviour, while showing a decline in daily signups does. When only 2% of your users are still using your product after 3 days, something is seriously wrong. A metric can be comparable and understandable, but as long as it does not make you see things different and behave accordingly, it is not a good metic (for you).
Remember, metrics need to uncover the pain in your startup. By showing you the pain in your startup, they trigger you to look for the actual problem. Metrics are also an essential part of the Lean Startup methodology. Every experiment has defined success criteria that need to be comparable, understandable and behaviour changing to work.
Next time we will dive into AARRR, an analytics framework for startups. Avast, me hearties!
Originally published at next.amsterdam on December 9, 2015.
