The Only Three Metrics That Really Matter For Startups
This is taken from Chapter 8 of my new book, “Explosive Growth — A Few Things I Learned Growing To 100 Million Users,” available on Amazon.
Explosive Growth is about the wildest startup story you’ve never heard of (until now). This compelling and inspiring narrative gives you a step-by-step playbook to achieve explosive growth, combining hilarious storytelling, proven tactics, and case-studies.
After a couple of years of non-stop explosive growth, which included acquiring 100,000 users in one day, the growth of our dating app, AreYouInterested (AYI), began to suddenly slow down. This wasn’t a big concern, however, as our focus had shifted from user growth to revenue growth. And so far the strategy was working. We were able to grow revenues from $3 million to $19 million in just two years due to our new subscription model, which even ranked us as the 36th fastest growing tech company in North America according to Deloitte’s Fast 500 Technology List.
However, this focus on revenue growth at all costs was causing some other key metrics to suffer. And soon after, the revenue growth itself suddenly and unexpectedly plateaued. At that point, we were very data driven, so collecting and analyzing pertinent data seemed like the best approach to begin solving our problem of diminishing growth.
We were measuring and trying to optimize thousands of metrics, which inadvertently caused us to lose focus on the reasons we were successful — having a remarkable and unique product that users loved to interact with and talk about. We needed to get back to the basics and figure out where we stood with them to understand how to create a new growth rocket. The three critical questions we needed to answer were:
- Was our product still remarkable?
- Did people love our product so much that they were telling others about it?
- Were users coming back over and over to use the product?
It’s easy for companies big and small to get lost in all the data and lose focus on these three key questions. These three insights are crucial, because they are actionable and predictive of future success, provide invaluable insights with only a handful of users, and are relevant at any stage of your product life cycle.
However, if the product isn’t unique, and people don’t love it (and don’t keep using it), at some point, you will start failing and it may be difficult to comprehend the reason. On the other hand, if these three questions can be answered with favorable results, all the other metrics will fall in line, and success is imminent. I discuss some other crucial questions that will help your startup succeed in “The 3 Questions Every Startup CEO Needs to Ask to Reach 1 Million Users.”
Fortunately, it was pretty easy to measure these things and get the relevant answers we needed.
The three questions above correlate to the only three metrics that really matter:
- Whether or not a Unique Selling Proposition exists (USP)
- What Your Net Promoter Score is (NPS)
- Do You Have High User Retention
Obviously, there are numerous other metrics you’ll need to measure in order to manage and grow your business including growth, engagement, and profitability metrics. The problem with most of them is they do little to tell you why your product is underperforming. The reason USP, NPS, and retention are the only ones that really matter is that they tell you why your product is underperforming, and hopefully give you the insight you’ll need to fix it.
For example, let’s say you look at poor growth or profitability metrics alone, and start firing underperforming employees. Then you begin thinking about new ways to grow and creating different marketing ideas. However, this will likely be a fruitless exercise, because the problem is most likely that your product simply sucks, and no amount of viral growth hacks, new talent, or new marketing tactics will overcome that. Whereas, if your NPS isn’t good (meaning nobody wants to tell their friends about your product), then the answer is clear: your product sucks! The mystery is solved. Unfortunately, in that case, you have a lot of work to do to fix things, but at least you know where the problem is.
Metric #1: Unique Selling Proposition (USP)
Is the product remarkable? When our dating app, AreYouInterested (AYI),was new on Facebook, there were several remarkable things about it, such as seeing which user’s friends liked them, and near-instant signup and profile creation. Now, we had to determine if we were still remarkable enough to stand out from the competition, and regain that elusive and extremely valuable word-of-mouth growth. To measure this, we decided to conduct a simple survey.
Part of the survey included a key customer satisfaction indicator known as the net promoter score (NPS), which the next section covers in more detail. Also within that survey, we included some other key questions related to the overall quality and uniqueness of the customer experience, such as:
- Which features of AreYouInterested do you use?
- What is your biggest frustration with AreYouInterested?
- What would make you more likely to tell your friends about AreYouInterested?
- Do you have any suggestions to make AreYouInterested better?
- What is the one sentence that best describes AreYouInterested?
That last question was crucial, because it told us if the majority of our users were experiencing something magical about the product. Users must come away with a singular message they’ll share with friends, otherwise, the message will never stick and spread. For example, Amazon’s original unique offering was to offer any book cheaper than the competition, and it worked beyond perfection.
We needed to know not only if our product was unique and had a great customer experience, but also if our branding was working.
Although there is no hard and fast rule about what comprises a good metric for it, I’d argue that a great USP is when at least 50 percent of the users identify the same remarkable item that best describes the product in one sentence.
#ExplosiveGrowthTip: Can you describe your product’s USP in one concise sentence? Is it truly remarkable?
#ExplosiveGrowthTip: Have you asked your users to describe your product in one sentence? Did at least half of the responses refer to the same concept?
Some of the best marketing books for entrepreneurs whose lessons are detailed in Explosive Growth are: Made to Stick: Why Some Ideas Survive and Others Die by Chip & Dan Heath, Contagious by Jonah Berger, Pyromarketing by Greg Stielstra, and Purple Cow by Seth Godin.
The results of that one question, “What is the one sentence that best describes AreYouInterested?” were eye-opening. We got answers that were all over the map, which meant we didn’t have one magical thing — a truly unique selling proposition — that customers fell in love with. That result meant either everything was remarkable to some users, or very little was remarkable to others. In our case, it was clearly the latter. This explained why our growth, especially organic growth, was stalling.
AreYouInterested was no longer remarkable. We would have to go back to the drawing board, because I didn’t see long-term success without regaining the ability to grow organically. The cost of acquiring users on Facebook was increasing by the day, because hundreds of millions of dollars had been raised by other companies — some much bigger than ours — that were building Facebook apps.
As a result, we spent a full year innovating to rebuild and relaunch as a “social discovery site” where users would meet new people through mutual friends and similar interests.
Friends of Friends (of Friends)
Although we were the first dating app to integrate mutual friends, most of our competitors quickly followed and integrated the concept of mutual friends, because Facebook made that very easy. It was an enormously popular feature, and we thought we could make it even better and help us stand out from the crowd by expanding it to friends of friends of friends. Unfortunately, Facebook didn’t provide this sort of information, so we had to build the feature ourselves.
Because it was truly a “big data” undertaking, using technologies unfamiliar to our company, I envisioned having the bulk of my team working on this feature for months. However, an unproven (but ambitious) engineer named David Fox boldly said he would take the lead on the task and deliver results. Having already witnessed his passion for his craft, and learning from past experiences, I gave him the chance to build something that would sing.
Ultimately, my faith in him was rewarded, because in just weeks, he almost single-handedly built something that matched up billions of social connections. Once again, the power of a terrific engineer — A-list talent — led to extraordinary results.
“Someone who is exceptional in their role is not just a little better than someone who is pretty good … they are 100 times better.” — Mark Zuckerberg, Cofounder of Facebook and internet entrepreneur
The relaunch was a sizable undertaking for our organization. We became the first company to introduce the “friends of friends” concept in online dating. A user could meet new people through mutual friends and friends of friends of friends, or, in Facebook terms, meet singles through a user’s social graph. Just knowing users had a friend in common — even if it was a second-degree relationship — was important to singles, especially women.
Through this relaunch, we had created a powerful new way to connect people, something nobody else was doing. It looked like a great feature, so we ran the same survey asking the same question once again a few months after launch: “What is the one sentence that best describes AreYouInterested?” The numbers didn’t budge — people still didn’t have a salient conception of our product. I began to worry that the entire rebranding around friends of friends was a huge was of time and a bad idea.
You Don’t Get a Second Chance to Make a First Impression
We learned a very painful but valuable lesson from that survey: it’s very difficult to change what a product is in the eyes of the existing user.
We implemented a bunch of new features that definitely added to the user experience, but nobody recognized that. Even more frustrating, several other new dating sites were getting very popular and growing rapidly with the same features.
A product named Hinge used the same concept of meeting people through mutual friends, and it gained great notoriety for it. It became the core of their brand, because they arrived on the online dating space with it. That feature wasn’t associated with our brand, however, because people already identified us with their first impression of us. We were “The Facebook Dating App” — the original — and that’s how people were going to think of us, no matter what we did with AreYouInterested going forward.
The other thing that hurt us was we had become a paid app, and those new apps were all free. At that point, Facebook users weren’t paying for apps or content within apps. All those new features we added were great, but they didn’t do anything to change our brand, solve our problems, or improve our bottom line. Sadly, the most eye-opening result of that survey came from answers to the crucial survey question: What is the one sentence that best describes AreYouInterested? Most users said we were the Facebook dating app that costs money.
It didn’t help that start-ups were raising tens of millions of dollars in funding, and a new competitor was offering a free product seemingly every day. Our biggest problem was like it or not (not), our paid business model had become our identity, and essentially our USP (Unique Selling Proposition).
Metric #2: Net Promoter Score (NPS)
“You can market your ass off, but if your product sucks, you’re dead.” — Gary Vaynerchuk, Hall-Of-Fame Entrepreneur and four-time best-selling author.
The NPS is an underutilized yet incredibly effective way to measure customer satisfaction. Our survey asked how likely users were to recommend our product to a friend or colleague on a scale from zero to ten. The responses were evaluated as follows:
- A response of nine or ten means a user is a “raving promoter of your product.”
- A response of seven or eight means a user is a “passive promoter of your product.”
- Any rating from zero to six means a user is a detractor.
Why Be So Hard on Yourself?
On the surface, it may seem a bit unreasonable to require a rating of seven or above for a user to be considered a promoter. However, the survey was trying to establish the potential for users to “actively promote” our product. If someone rated our product a six, they likely thought it was a decent product — above average — but they weren’t likely to rave about it to friends. On the lower end of the scale, it didn’t really matter if someone rated the product a zero or a three, because either way, they didn’t like it. That person was a detractor, and there wasn’t much we could do to change their mind.
On the other hand, if someone rated the product a nine, that meant they thought highly of the product, and were likely to rave about it to their friends and family. It was likely the product was something they wanted to bring up in conversation, because they found it so unique and interesting — definitely magical, a true USP.
In its early days, Tinder spread like wildfire due to several unique features, brilliant marketing, and a 10x product experience that had everybody talking about it. Every startup can learn from Tinder’s product and marketing strategy and execution which I analyze in the 5 genius things Tinder did to achieve explosive growth.
#ExplosiveGrowthTip: Ask if users have already recommended your product to a friend. Do you know what percentage of your users have recommended your product to a friend? If they said no, try to find out why. Perhaps you just haven’t made it easy enough for users to share your product — an easy fix.
To calculate your NPS, simply subtract the percentage of detractors from the percentage of promoters. The NPS score ranges from -100 (everybody is a detractor) to +100 (everybody is a promoter). An NPS above zero (indicating more promoters than detractors) is desirable, but an NPS above fifty is viewed as an extremely positive indicator. Unfortunately for us, we learned that our NPS wasn’t very good, which explained why our organic growth had slowed.
The combination of a less-than-stellar NPS with some unfavorable responses to our additional questions was a rude awakening for us, but it also provided clarity as to why certain metrics were performing so poorly. As stated previously, surface metrics won’t usually reveal big-picture branding problems, such as the product being boring and ordinary. You may be winning the battle with short-term optimizations, but losing the war with a product that still isn’t unique and valuable enough for users to tell their friends about.
The data told us we still needed more innovation to achieve a true USP and a great NPS. We needed to make our product remarkable (again) so users would want to spread the word. The mutual friends concept, and even the second-degree-of-friends idea wasn’t going to be enough. We needed to go back to the drawing board at least one more time.
#ExplosiveGrowthTip: Are you actively measuring your NPS? Is your NPS above fifty? Don’t waste time and money by trying to market and grow a product that has a poor NPS.
Early-stage companies are obsessively focused on growth, usually at the expense of retention. That methodology is backwards, because retention should be the north-star metric of any early-stage product. If someone builds a great product that users keep coming back to, it will be easy to figure out how to grow it (raise money, spend money on acquiring users, ask users to share it, etc.) However, if a product has a low NPS, and you continue to focus on growing it, that’s like saying to the user, “Hey, my product sucks. Want to buy it?”
#ExplosiveGrowthTip: Marketing a product with a low NPS is essentially saying to potential customers, “Hey, my product sucks, come check it out.”
Metric #3: Retention
Ultimately, retention is the most important metric to measure for any online business — if people keep coming back to the product, every other problem becomes of little significance. Establishing AreYouInterested’s initial user base was valuable, but we could always pay users to try it. We couldn’t, however, pay users to keep coming back, day after day, week after week, and month after month. An understanding of what drove users to continue to use the product repeatedly was the most important insight we had. A strong retention rate is irrefutable data that proves people love your product, and I believe the same goes for most businesses.
What drives people to continuously use an online dating site is a great user experience — that magic moment of getting messages (and ultimately dates) from people they want to meet. Getting a reply to an email from a potentially special someone on an online dating site will keep that user coming back, over and over. Of course, if that reply leads to a serious relationship, then to a commitment, then maybe the dating app did too good of a job; but that’s a whole different problem.
Facebook’s Magic Number for Retention
In the early days of Facebook, the leaders recognized a drastic difference in retention based on how many friends each user had in their first ten days. If a user had less than seven friends, it wasn’t interesting to them and they didn’t come back to the site often, as their newsfeed just wasn’t active enough to keep them engaged. However, if a user had seven or more friends within ten days after signing up, the retention rate was very high, as the newsfeed seemingly came to life. It’s probably a much different number now, but seven was their magic number in those days and early Facebook employees implied this was a watershed moment for them.
Another key metric that Facebook was obsessed with was getting 90 percent of its users to login six out of seven days per week. Obviously, any product that gets that kind of retention is going to be wildly successful.
The lesson here is that not only is it crucial to measure retention, but it’s also crucial to understand what ultimately drives users to come back to the product — your product’s “aha” moment. This can be figured out by separating out the high-retention users from low-retention users and analyzing the data to look for what makes the high-retention users different from others. Do they have more friends (Facebook)? Do they have more replies (AreYouInterested)?
That insight alone can change the business, because it explains what to optimize for.
For us, it was obvious to see in the data that high retention users got lots of replies, and users who had few replies didn’t bother to come back. With that type of insight, we decided to eliminate all goals except one — to double the number of replies. We spent the next 90 days focused exclusively on this one goal, and this laser-like focus enabled us to increase replies by 400%, which correlated with an increase in revenues and retention.
We were able to enhance the experience for the users who were lacking replies, including improving our algorithms to surface better potential matches and building features such as “Priority Placement,” where they could pay for increased exposure in order to get more replies, hopefully leading to higher retention.
#ExplosiveGrowthTip: Do you know what single user action or experience compels users to come back to your product repeatedly? If not, figure it out now, because this could be the most important insight you need to grow your business.
Once Facebook discovered that “lucky seven” was their goal, they focused serious labor on increasing the number of friends on every user’s profile. They did this by suggesting friends based on data that nobody else was thinking about. Users logged in and saw friend suggestions based on a fourth-grade classmate, a person their third cousin met at a bar in Albuquerque a few years ago, and other previously unrecognized variables. They immediately become enamored with a connection to a potentially long-lost friend or the excitement of connecting with someone on the outer circle of their life.
The ideal retention metric for any online business should be based on how frequently a user comes back, what percentage of users come back in a certain time frame, or any combination thereof. Generally, one, thirty, ninety, and 360-day data should be sufficient to gauge retention rates, but it’s still important to discover why users keep coming back.
Further Reading: Legendary “Product Guy” and Venture Capitalist, Josh Elman, has some invaluable product insights regarding these metrics. Elman, who has incomparable experience and success leading product teams at Facebook, Twitter, and LinkedIn, explains in his post, The Only Metric That Matters, that you must find a metric that serves as a “signal that (users) are using the product in the way you expected and that they use it enough so that you believe they will come back to use it more and more.”
#ExplosiveGrowthTip: Growth without retention is worthless. However, retention without growth is a problem any entrepreneur should love to have, because it means people love the product. Do you know what your one-day and thirty-day retention is?
#ExplosiveGrowthTip: Don’t spend significant money on marketing until your one-day and thirty-day retention is well above average for your industry. View my Top 10 Explosive Growth Tips For Entrepreneurs.
About the Author
Cliff Lerner is the Founder of Snap Interactive (now PeerStream), the first publicly traded social media company, and BuySellHodl — a Bitcoin Price Prediction App. Cliff recently published the Best Seller Business Book, Explosive Growth: A Few Things I Learned While Growing To 100 Million Users & Losing $78 Million, which can be purchased on Amazon. Contact Cliff if your business needs help growing.
