Book Summary: Lean Analytics

A step-by-step guide on how to use data to make your startup take off

Who should read this book?

You either have a startup, are thinking of starting one, or work as a data analyst at one. Either way, you want to use analytics to make your life a bit easier and the business grow.

One-paragraph summary

The book defines the most important metrics for startups, depending on their business model and at what stage they are, alongside with baselines for these metrics. Then, it gives you tips to help you use analytics wisely, ideally throughout the company’s whole life, without getting lost in too much data, by always keeping in mind what he calls the “One Metric That Matters”: at any given time, the one metric that you should worry about the most, above all others.

Full summary

Lean Analytics is sort of a Lean Startup spin-off. The “Lean” workflow is about learning, building and measuring. While the Lean Startup is about structuring progress, Lean Analytics is about measuring progress. It’s almost a roadmap for those willing to build a data-centric startup from scratch.

The book starts by teaching you how to identify the most important metrics for your company’s business model and stage of development. Then it defines some baselines for this metrics. It ends with a couple of (dispensable) chapters on specificities related to B2B startups and intrapreneurship. It wraps it up with a conclusion that also doesn’t add much. The whole book is also permeated with general tips related to business and data, some of them quite useful.

What metrics are relevant for your company’s business model?

“Marketing is about selling more stuff to more people more often for more money more efficiently.” — Sergio Zyman, Coca-Cola’s CEO

E-commerce

When you have an e-commerce, look at the annual repurchase rate (out of everyone who bought something last year, what % will do it again this year?). If this number is under 40%, you are in acquisition mode, if it’s between 40% and 60%, you’re on hybrid mode, and if it’s over 60%, you’re in loyalty mode. Ok, but what does that mean? If you’re in acquisition mode, your focus should be on new customer acquisition. If it’s on loyalty mode, focus on loyalty, obviously (keep your customers spending money on your business). And on hybrid mode, do a little bit of both.

If thinking in terms of years is too much for your young company, look at the 90-day repurchase rates: less than 15% means acquisition, between 15% and 30% means hybrid, and over 30% means loyalty.

Remember knowing this is vital for your business, as it shows you where to focus to get revenue. By the way, you can’t dramatically change these percentages.

Finally, the One Metric That Matters for you, regardless of the mode you are in, is revenue per customer: it mixes conversion rate, repeat purchases and transaction size.

Software as a Service (SaaS)

Not much to see here: you want to measure engagement and churn, both on a different time frames: daily, weekly, monthly and yearly.

Free Mobile App

Keep track of average revenue per user (or paying user) and Customer Acquisition Cost (CAC). Also, keep an eye on whales: those users who spend A LOT of money on your app and should be a segment on its own.

Media Sites

For revenue, try displaying ads, affiliate links, and pay-per-click ads. Watch your audience and churn, ad inventory, ad rates, click-through rates, content/advertising balance.

User-generated Content

There’s a funnel, like in e-commerce, but this one is based on user’s interaction: it’s called an engagement funnel. This engagement is everything for User-generated content businesses.

To encourage it, notify users constantly by app and email (not so much that they will hate you).

Two-Sided Marketplaces

Here, there’s a “chicken-egg” problem: who comes first, buyers or sellers? Short answer: buyers. Create supply artificially first, to attract buyers. Once they are there, sellers will follow the money, allowing you to reduce artificial supply.

Metrics that matter: buyer and seller growth, inventory growth, search effectiveness, conversion funnels, ratings, signs of fraud and pricing.

What metrics are relevant at your company’s stage?

The book gathers a lot of concepts from other authors to create its own version of the stages of development a startup goes through, in increasing order of size and complexity.

Empathy (identifying the problem and the solution)

It’s the starting point for you: you have spotted a possible business opportunity, and have decided to investigate a bit further. At this stage, you want to make sure the problem is worth solving: it’s painful enough, enough people care, and they are already trying to solve it.

To find these answers interview at least 15 people, using a script, and get qualitative answers. Here’s a script suggestion:

• Set the stage (introduction)
• Customer segment (get demographics)
• Set the problem context (tell a story)
• Test the problem
• Test the solution
• Ask for something (planting the seed for future sales)

On top of that, run some rough estimations to try and guess how much money is in that market: how many customers could you get a month? How much do they spend, in average?

Then, scale the interview process, by running surveys. Once you are sure that you understand your customer well enough and that you’ve found a problem worth solving with a solution that meets the market’s needs, you’re ready to move on.

“Don’t sell what you can make; make what you can sell.” — The book

Stickiness (making sure your product is good enough)

Now you want to make sure your product is right for your customers: the minimum viable product (MVP) is actually not a product, but a process, so iterate over it to focus on retention and engagement. You do this by rigorously testing new features and how they affect retention and engagement.

You should only move to the next stage once your customers are using your product as expected and getting enough value out of it.

“If we are building the wrong product, why should we be proud of doing it on time and on budget?” — Eric Ries

Virality (spreading the word)

This is where most growth hackers do their job: they find a metric that causes virality in the future, and try to change it now. Virality can come in three forms:

Inherent, when it’s a built-in feature of your product, for example when you can share your runs from your running app to Facebook. It’s the best kind of virality.

Artificial, when you “pay” for your users to share your stuff (for example, when Dropbox gives you extra space for tweeting about their product).

Word-of-mouth, which is also a very powerful asset, but on which you don’t have much control: just make sure your product is good enough for people to talk about it.

The metric to watch here is the viral coefficient: how many users does a user bring? Divide “accepted invitations” by “existing customers”. Ideally, it will be above 1, but that’s not always the case. There’s also another metric that matters here: cycle time. How long does it take for a customer to invite those people? You should also be able to mix both: “how many users does a user bring in X days?”.

For B2B, look for the NPS: the comparison between users who are very likely to recommend your product vs. those who are very unlikely to do it.

The book doesn’t present a clear sign that you can move on from virality: once you feel your product or service is viral enough, and this virality is sustainable, feel free to move to the next stage.

Revenue (making it profitable)

Now, the goal is to maximize the ration between the money customers bring in (Customer Lifetime Value) and the cost to acquire them (CAC). Maximizing CLV/CAC also means attacking the breakeven point: the earlier a customer expenses overcomes the cost to bring her, the better.

Once you have reached the revenues and margins expected on your business model, that are enough for your company to achieve sustainable and profitable growth, move on to the next and final step.

Scale (making it sustainably profitable)

Scaling is about the health of your ecosystem: focusing on improving the efficiency of you channels and your traffic and understand where you are, compared to your competitors. It’s all about making sure you can grow consistently, fast and sustainably. Now, obviously, there’s no next stage: you will just become a big, boring company. Sorry.

What metrics are relevant for you?

Below we have the main metrics for each combination of business model and stage:

What are some baselines for those metrics?

Overall

Growth rate (in terms of users or revenue): 5% a week, in very early stages

Engaged visitors: 30% visit once a month, 10% visit daily

CAC/CLV: 1/3

Virality coefficient: 0.75. Ideally, above 1

Mail lists: 20–30% open rate, 5% click-through

Uptime and reliability: for a critical paid service, at least 99.5% of uptime. For the rest, it can be a bit lower

Site engagement: 1 minute per page

Web performance: less than 5 seconds to load a page. No more than 10 seconds.

E-commerce

Conversion rate: 1–3%

Cart abandonment: 65%

SaaS

Revenue growth rate: 20%/year

Increase on what your current users pay: 2%/year

Max churn: 5% (ideally 2–3%)

If you don’t ask for a credit card upfront:

• 10% of visitors try the product
• 25% of those use it

If you ask for a credit card upfront (usually, it’s better not to):

• 2% of visitors try the product
• 50% of those use it

Free Mobile Apps

Mobile download size: try your best to keep it under 50Mb

CAC:

• $0.50 for a paid (mercenary) install
• $2.50 for an organic one,
• $0.75 overall. Never more than the CLV.

Application launch rate: 83% is already good

Churn: up to 80% on first day and 98% on first month

% of paying users: 2% for freemium apps, 1.5% for in-app purchases

Average revenue per daily active user: minimum of $0.05

Monthly average per daily active user: $3 ($0.10 per day)

Average revenue per paying user — segmented by their spending, split into categories according to their “size”:

• Whales: $20 (10% of users)
• Dolphins: $5 (40% of users)
• Minnows: $1 (50% of users)

Ratings click-through: less than 1.5% for paid apps, and around 0.07% for free apps

Media sites

Click-through rates: 0.5–2% (never below 0.08%)

Sessions-to-click ratio: 95%

Engaged time: 90 seconds for content pages, 50 seconds for landing page

User-generated content

Time on site per day: 17 minutes

Engagement:

• Lurkers: 25%
• Casual contributor: 65%
• Highly engaged: 10%

Two-sided marketplaces

There are no baselines for two-sided marketplaces, you can use the ones from e-commerce sites. When you have no baselines, see how are your results converging: the more you improve a metric, the harder it gets to improve it further. When you think your metric has reached a plateau or is good enough, stop trying to improve it.

Key take-aways

Here are some general insights provided by the book, that don’t fit a specific category (except the “Key take-aways” category, obviously):

When you have a hypothesis, test it quickly and with minimal investment, defining your goals beforehand (key metrics and what’s the desired improvement).

A good metric is

• Comparative
• Understandable
• Usually a ratio or a rate
• Actionable

The Lean Analytics Cycle

• Choose a KPI that you want to improve and set a goal for it
• Find features that might affect this KPI
• Test different versions of that feature
• If it works, great! If not, try again or give up.

Instilling a data-driven culture in your company

• Start small, pick one thing and show value
• Set clear goals
• Get executive buy-in
• Make it simple to understand
• Make it transparent
• Don’t underestimate the importance of instinct
• Ask good questions

B2B tips

• Your customer base will be a lot smaller, so you won’t be able to iterate and test as much. On the other hand, it’s easier to get deeper insight from all your customers.
• Make sure your solution integrates well with customers’ existing tools.
• Try to start with consulting services first, as it will allow you to better understand your market, and build a customer base to eventually buy your products in the future.

Intrapreneurship tips

• Your first step will be getting executive buy-in: having an executive level sponsor is crucial to succeed and innovate in a bigger company.
• Insist on having access to resources and clients, without having to “resell” the idea midway all the time.
• Keep your reports simple and honest.
• Leverage on scale, whenever possible.
• Agree on goals and general rules before starting the project, and make sure they are clear to everyone.
• Limit access to the team from outsiders.

“Your startup has succeeded when it’s a sustainable, repeatable business that can generate a return to its founders and investors.” — Also the book.

Conclusion

Overall, it’s a good book, but it could be a lot shorter. Once you understand its structure, it’s a good guide to keep nearby and go to in case you need some insights. It’s very specific and startup-oriented, however, so not worth it for those who are not part of that world. It also has some examples and case studies that might give you some good ideas for your business. I’d give it a 8/10.