Startups, Metrics, and Intellectual Honesty
VCs writing blog posts explaining how to calculate some startup metric has become a familiar trope. These posts can be helpful, and some of them are truly insightful, but they mainly deal in the science of calculating these metrics. The truth is that compiling any set of metrics is both science and art. The reality is that any metric can be made to say anything based on what inputs are used in the formula. But there is no point in measuring (and managing to) a metric if it isn’t accurately quantifying the business.
Recently, this came up with a company I have been informally advising for a few years. This company does a great job of measuring and quantifying a number of aspects of its business. However, the company had historically omitted the cost of customer support from its CAC and LTV calculations. While customer service is often placed below the gross margin line in many cases (such as this company’s) it may scale directly with revenue in which case it most certainly impacts contribution margin which drives LTV.[1] The problem with not including these costs, is the company was not actually measuring what it thought it was measuring, and if you’re using these metrics to make strategic decisions, then you risk coming to the wrong conclusions.
Unfortunately, this is a trend we’ve seen a lot in recent years. Lifetime Value (LTV) and Customer Acquisition Cost (CAC) formulas and ratios tend to be the greatest offenders. Everyone calculates them differently, and given the host of inputs that go into them they are easy to manipulate. Bill Gurley has an excellent post explaining why these calculations are not the end-all-be-all startup metric, but rather simply one tool in measuring the of health of your business. Many companies report LTV as simply lifetime revenue — choosing to wholly ignore the issue of margins, which is fine if you think of it as revenue rather than profit (and admit to yourself that it doesn’t really tell you much). In other cases, we’ve seen entire COGS line items omitted, or “future contribution margin” used despite it being 10 points higher than “current contribution margin”.
Intellectual honesty is the foundation to any attempt to measure and quantify the progress and success of one’s business. You have to be willing to simultaneously see yourself for what you are, as well as what one day you will become. Venture backed startups are locked in a race against the clock. You must grow fast enough to attract future financing or become profitable — otherwise you die (Paul Graham has a really good post on this). From the time you raise your current round of financing you have 18–24 (on average) months to achieve one of those two things. This leaves very limited time for course corrections if they need to be made. This is why it is critical that you (and your board) force yourself to be completely honest with yourselves as you measure the health of the business. How much can you spend on marketing to generate a dollar of contribution margin? Do you really believe the customer lifetime is what you say it is? You want to avoid being 9 months into your 18 months of oxygen thinking the business is doing great only to realize that you’re selling a dollar for $0.95.
At this point its worth acknowledging that there can be a difference between “marketing” metrics and dashboard metrics. For the purposes of fund raising you may choose to present your business in the most positive light possible, and let an investor come to their own view. This is fine, just realize that a good investor will do their own work and what you show an investor impacts an investor’s point of view of management.
Your early stage metrics don’t have to be perfect. Plenty of very large companies had terrible economics early on (for instance, early on LendingClub’s CAC was greater than revenue on a given loan). What’s important is to show that a market exists for your service/product and understand what it takes to get there. Good early stage investors (and board members) want to see growth and that you understand where you are and have an understanding of what it takes to get where you want to go (and that the economics are attractive once there). It is ok to have sub-optimal numbers now if you have a convincing story of how the metrics improve as you scale (network effect, economy of scale, etc). What is scary is a company that doesn’t realize where it is (thinks its core business is much healthier than it is) and can’t articulate (and may not be aware of) the bet they are making on some effect of greater scale kicking in as they get larger.
The basis for any mutually beneficial long-lasting relationship is honesty. This honesty has to start with yourself.
[1] My broad POV here is that if any cost scales directly with revenue it should be include in CM for these purposes.
