For the Lean (and Lost) Startup… [Uncut Version]
An answer to the three questions that plague lean startups
Short on time? You can find the short-form version of this post here.
Lean Startup has flourished in the last decade, now a common term among entrepreneurs and corporate innovators alike. In short, Lean Startup asserts that startups are essentially research organizations and that the founders’ primary goal should be to conduct low-cost experiments to accumulate “validated learning” about their customers’ needs and behaviors and the viability of their business ideas.
In my work interfacing with hundreds of entrepreneurs at Stanford and within accelerators around the Bay Area, I’ve noticed that many innovators following the Lean Startup approach get stuck in a few key areas:
- They don’t know how to prioritize the many hypotheses they want to test.
- They don’t know how to distinguish positive signals from negative signals in their experiments.
- They don’t know whether they are actually making progress in creating a viable venture.
In this post, I outline a new framework called “The Checkpoints of Validated Learning” that I developed to help Lean (and lost) Startup entrepreneurs and corporate innovators get “unstuck” in their venture development process.
To help explain the framework, I will refer to an example entrepreneur called “DeeDee”.
OK, on to The Checkpoints
At each Checkpoint in the framework below, if your feedback is negative, you’ll need to conduct Creative Problem-Solving (this is part of the “art” of entrepreneurship that I was talking about above) to iterate until you can clear the Checkpoint and move onto the next one.
(In a future post, I’ll discuss how teams can boost their Creative Problem-Solving ability, so Follow me to get notified when that post is out).
Disclaimer: “Entrepreneurship is an Art, Not a Science”
When I present this framework to seasoned entrepreneurs and investors, I sometimes get the reaction “but Keegan, entrepreneurship is an art of exploration, not a deductive science or a process diagram”. I agree 100% with this sentiment, and that’s exactly what makes the framework here so helpful. Let me explain:
Science as a practice is actually not about following formulas, process diagrams, and step-by-step instructions. People associate science with these things because that is how people learn about science.
In science class, you experiment according to protocols, and you apply equations and formulas. But after all that schooling is done, after you’ve learned all the formulas and rules that mark the pinnacle of human knowledge, then you can use them as tools to actually practice science.
Practicing science is an extremely creative endeavor, an exploration into the unknown, one that involves continuous problem solving, late-night brainstorms, operating cooperatively and competitively with others in the face of high uncertainty, and constantly worrying about fundraising (sound familiar?).
It is in this light that I present the framework here. Think of this framework not as “an almighty checklist of entrepreneurship”, or rigid rules to be bound by, but rather a tool to help you focus your energy on the art of entrepreneurship which requires continuous Creative Problem-Solving.
The Starting Point
The starting point of this framework is that you have developed a hypothesis that a specific “user-need” exists, which consists of both a specific “user” group, and a specific “need” that your specific user group has.
Some entrepreneurs will do 100 user interviews before settling on a specific user-need. Others may have a very strong hypothesis of a user-need with just a few interviews because they’ve felt the need themselves.
For example, our intrepid entrepreneur, DeeDee, wanted to help small, independent restaurant owners. After doing dozens of open-ended interviews with these owners, she discovered a common theme that they were missing out on orders because they didn’t offer delivery. By listening to and observing her users, DeeDee developed a “user-need” hypothesis that independent restaurant owners (user) want to increase their revenues by offering delivery, but that hiring delivery drivers to do 1–2 deliveries per day is difficult (need).
Checkpoint #1: Is your User-Need valid?
No matter what work you’ve done previously to establish a user-need, you’ll want to use a prototype to validate it. To be clear, by “prototype”, I mean some tangible manifestation of your hypothesis. This can be something as simple as a pencil sketch on paper of what you believe is the user’s current pain point (i.e. a simple storyboard).
At this early stage, you don’t need to invest time and money in creating the famous “MVP” (Minimum Viable Product) and in fact, there are even advantages of using the lowest-cost method at each Checkpoint, which I will talk about in a future post.
(Pro-tip: Including emotions in your sketches is helpful because you want to test that you understand what specific emotions are felt by the user experiencing the pain point).
In our example, DeeDee draws a sketch of a frustrated restaurant owner having trouble finding delivery drivers willing to do only 1–2 deliveries per day. She presents this to several restaurant owners and gains the insight that it’s not actually that difficult to find contract workers willing to do the work, but rather the restaurant owners have a hard time trusting these contractors to fulfill the deliveries and give their customers a good experience. DeeDee redraws her sketch of the user need (this can be done live during the interview of course) until her interviewees give her the positive indication that this is the specific pain point they feel, including specific emotions (e.g. distrust of strangers, worry about reputation, etc.)
This is intentionally placed as the first Checkpoint because:
- This is one of the most common sources of error I see among early-stage entrepreneurs, who spend months testing ideas that don’t resonate because they never accurately pinpointed the user-need, and
- It is easier to clear this Checkpoint the earlier it is done in your journey, when your emotional attachment to solution ideas is at its minimum and your ability to adapt to user feedback is at its highest.
The user feedback you get determines whether you proceed to the next Checkpoint, or if you need to iterate on your user-need (iterating on either the “user” or the “need”, or both).
You’ll see in the framework’s Evidence for “Yes” and “No”, that user feedback is often deceptively positive (especially if you know the person you’re interviewing), so be wary of those false positives. Entrepreneurs need to hear strong, emotion-laden feedback from their interviewees to determine if they’ve truly identified a valid user need.
Checkpoint #2: Is your Value Hypothesis valid?
After you’ve validated your user-need, you’ll next want to validate your Value Hypothesis (i.e. the value proposition you think will actually solve the user-need) by creating a prototype that illustrates a solution idea and demonstrates the value proposition that it will deliver.
(Pro-tip: I highly recommend testing at least two distinct value hypotheses at a time, each with a distinct prototype. I’ll discuss the value of this more in a future post.)
A simple paper sketch prototype is still very valuable at this stage, though you may wish to use other techniques to get the concept across (e.g. a promo video). In-person interviews are still the most useful validation mechanism at this stage, though some entrepreneurs will want to supplement those interviews with some basic A/B ad testing comparing different value propositions.
In our example, DeeDee draws out a sketch of a solution idea she has, wherein there is a pool of contract delivery drivers available through a service called Deliveries-R-Us. When the restaurant gets a delivery order, they transmit the order info directly to Deliveries-R-Us using software that integrates with their own order processing software. A Deliveries-R-Us driver then comes to pick up the food and deliver it to the customer. The feedback she gets from the restaurant owners is overwhelmingly positive, especially the fact that integrating the new software won’t require any new training or introduce any new opportunities for human error.
Like with Checkpoint #1, you’ll want to get strong, emotion-laden reactions from your interviewees to really demonstrate that the value proposition is resonating. Otherwise, you might be persuaded by false positives to build something that people won’t actually buy.
Put another way, take account of how much effort is needed to ‘convince’ your user that your solution idea will solve their need (or that they have the need in the first place). Does it take 3 seconds or 3 minutes? Taking 3 minutes may not seem like a big deal in the interview setting, but imagine that it will then take the equivalent of 3 minutes of direct attention of your customers for each sale! If you’re lucky enough to capture 3 minutes of someone’s attention, your customer- acquisition cost is likely to be prohibitively high for that solution idea.
Checkpoint #3: Does your MVP actually deliver the Value Proposition?
Now it’s time to build the famous Minimum Viable Product, or “MVP”, the lowest-fidelity prototype of your solution that actually delivers the value proposition to the user. It’s not uncommon for a Value Hypothesis to be validated in Checkpoint #2, but then once the MVP is built, the founders discover important criteria needed to effectively deliver that value.
In our example example, DeeDee decides to perform a “Concierge Prototype”, a form of MVP wherein she performs the complete value exchange of her solution idea with one customer. DeeDee’s plan was that she could make money with her solution by charging the end consumer (i.e. the burrito eater) a little bit more than the delivery costs. She puts up some flyers around her campus for “Bob’s Burritos Now Delivers for $5 — Call Now!”, and includes her cell number. Soon she starts getting calls from strangers for burrito deliveries (hurray!). She takes their order and tells them how much they owe. She then calls Bob’s Burritos to submit the order for pick up. She goes and picks it up, pays for it with her own money, and then drives it to the customer that called her. They pay her the amount she paid to Bob’s Burritos, plus an extra $5 for the delivery. Woohoo! She has now completed a full value exchange with a customer.
Note, it’s completely fine that the process was extremely inefficient at this stage. What she learns is that the restaurant is happy (they sold another Burrito with no change to their current operations), and the end customer is happy (they got their favorite burrito delivered by a friendly person for only $5). She does this a few more times to declare that her MVP does indeed deliver (pun!) her value proposition!
(Pro-tip: When testing your MVP, in-person user interviews are still very valuable. Entrepreneurs often release their MVPs to a wide audience to track the high-level numbers on who signs up. This is fine, but be sure to contact those users after they sign up and get them on the phone or in-person to provide their feedback. At this stage, you are still very early in your venture creation process, where in-depth commentary from users will provide you with the nuances of what is and isn’t resonating much more effectively than statistical data of the number of users who preferred version A or B.)
Checkpoint #4: Does your Revenue Stream have a viable CAC?
OK, so you’ve now got yourself an MVP that is resonating with users. You’ve accomplished what the vast majority of entrepreneurs and corporate innovators never will. Now you need to make sure you can actually afford to acquire customers for your solution!
This is the stage when you should focus on experimenting with different revenue streams and pricing tolerance, and keep track of the costs of delivering your solution (fixed and variable). Most importantly, you’ll want to run experiments to determine your Customer-Acquisition Cost (CAC) with your proposed revenue stream.
DeeDee performed her concierge prototype for a few dozen more customers. She tracked all the expenses she could think of, and she tested different delivery pricing to see what consumers would be willing to pay. DeeDee also performed supplementary experiments to validate other parts of her economic model, such as how much time is needed to recruit each new delivery driver. All of this modeling revealed that, even if she was able to automate everything about her solution, the cost to the end customer would be too high for most people, and would lead to a nonviable CAC. She applied Creative Problem-Solving to iterate on her Revenue Stream and pivoted to the restaurants themselves being the customer. She continued to conduct concierge prototypes, this time only for restaurants that signed up to pay her a service commission. The small restaurant owners she was working with recognized the value DeeDee was providing and agreed to give her a X% commission on all orders that were delivered through her service, plus an additional Y% for any order that she solicited through her own marketing efforts.
After signing up about a dozen restaurants through ads and performing about 50 deliveries with her new MVP, DeeDee now had sufficient evidence that her new Revenue Stream had a viable CAC. With this data, DeeDee secured some small angel funds and built out the software back-end to automate her earlier concierge prototype.
Checkpoint #5: Can you be profitable with this solution?
OK, so you’ve gathered enough data to validate that you can afford to acquire customers. Now you need to validate that you can actually create a profitable business with your solution.
For some ventures, Checkpoints #4 and #5 can be cleared simultaneously (e.g. your tests in selling your MVP provided you enough data to indicate that your MVP itself, or with minimal adjustment, would give you healthy profit margins). But for most ventures, the activities and timescales needed to clear each Checkpoint differ, and profitability only occurs after the venture has reached a certain scale. Many ventures will need to clear Checkpoint #4 to convince an angel or seed investor to provide them with enough financial runway to perform the larger-scale experiments for clearing Checkpoint #5.
The economics for profitability vary widely between different types of ventures, but one benchmark that is applied broadly at this stage is if the Long-Term Value (LTV) of each customer is greater or equal to 3-times the CAC.
Pro-tip: In the early stages of your venture, it is completely normal for LTV to still be laden with assumptions (after all, how could you know what your LTV is over 3 years when you’ve just started selling a month ago?), but just be sure to stay realistic and back up your assumptions with as much real data and analogous product analyses as you can.
In DeeDee’s case, after she launched her beta release of her software solution, she could finally complete enough value exchanges (say, 500 deliveries completed) to provide enough data about her customers’ behavior, and her revenue and cost drivers to have a data-backed estimation of her LTV and CAC. Her experiments have shown that she has a healthy-enough LTV/CAC ratio to create a profitable business with her solution (or at least a data-supported path for getting there).
OK, cool…so I now have a viable venture, right?
You’ll see that the framework ends with an open-ended task of identifying the next key risks to the business and conducting experiments to mitigate those risks.
These risks will vary wildly from venture to venture (regulation, technology, market shifts, etc.) and are so specific to the venture, that I won’t attempt to capture them in this framework.
You may be thinking “Wait a second! Aren’t I supposed to identify my key risks at the beginning of the process, and tackle the biggest risks first?” The reason this framework places these activities after Checkpoint #5 is that for most ventures (at least for the hundreds of entrepreneurs that I’ve worked with), validating that their solution idea can solve a real user-need in a way that can be profitable is the biggest risk to their venture!
The reality is most ventures won’t ever have the luxury of tackling other risks beyond this. Of course, if you’re dealing with anything medical or with highly sensitive data, you’ll want to address regulatory and security risks throughout the process.
So, why THIS order?
The order of the Checkpoints presented here was created from 2 guidelines: 1) It prioritizes what I’ve seen to be the areas of highest risk of time-wasting and missteps among entrepreneurs, and 2) it creates what I call “Safety Nets of Value” at each Checkpoint, which helps to de-risk the venture creation process in general.
By “Safety Nets of Value”, I mean that at each checkpoint, the entrepreneur is accumulating a set of knowledge, data, and/or assets that are valuable to others.
For example, let’s say an entrepreneur has gotten through the first 4 Checkpoints in the framework. She has validated a user need, she has validated a value hypothesis, she has created an MVP that delivers that value, she has validated that she can get an LTV/CAC ratio of 1, but she has found that she’s unable to get her costs down to be profitable with the solution (thus not yet clearing Checkpoint #5).
In this case, she has created a ton of value along her journey; value that she can leverage now (e.g. by attempting to get acquired or acqui-hired by a larger company that could bring her costs down, or by creating new technology or channels that don’t exist) or later (e.g. by waiting for a new channel or technology to appear in the market).
These Safety Nets of Value are particularly valuable in a corporate innovation context, where projects are impacted by tangential corporate activity. For example, support for a venture idea can come and go at the whim of the current focus of the CEO, projects can be delayed for years or canceled altogether for corporate branding reasons, explorations can be shifted to different project leaders and teams with different Creative Problem-Solving ability.
Creating value throughout the journey (and documenting it appropriately), greatly increases the chances that the journey will eventually lead to a fruitful outcome.
More to come…
In case you’re curious, our tenacious example entrepreneur, DeeDee, is based (with some deviation for illustration) on the origin story of DoorDash , a now multibillion-dollar food delivery business that originated in Startup Garage at Stanford
In this post, I’ve introduced a tool to help you decide what experiments you should test first (or next), to help interpret the feedback you are getting, and to help you track your progress in your venture creation process. I look forward to hearing if this tool has been helpful for you and your team, or if you think the framework needs adjustment or expansion.
Want to share highlights with others? You can find the short-form version of this post here.
In future posts, I’ll dive deeper into aspects of this framework and also other best-practices in entrepreneurship today, which I am lucky to be witness to in my work with the best founders and investors in the world.
(**Be sure to follow me to get notified when new posts are out**)
About the Author: Keegan Cooke serves as Associate Director for the Center for Entrepreneurial Studies (CES) at the Stanford Graduate School of Business (GSB). The thoughts expressed here are his own and not the GSB’s. Prior to the GSB, Keegan has served as a founder coach and innovation consultant, founded a STEM education startup, Magical Microbes (acquired) and served as Principal Research Scientist for a cleantech startup, Trophos Energy (acquired).
