THE LEAN STARTUP
5 Things Product Teams Get Wrong About the Lean Startup
An honest account on how The Lean Startup is being implemented in practice and why 90% of the Startups still fail
In my previous post, I gave a brief overview of The Lean Startup theory of innovation by Eric Ries, and walked through its key concepts. As Ries describes in the introduction part to his book, his motivation to write it came mostly from his observation of the troublesome rate by which startups all around him were failing .
According to him, these are the two key reasons why startups fail:
- Planning too much — what Ries calls “the allure of a good plan, a solid strategy, and thorough market research”; in a startup world of extreme uncertainty, this traditional thinking, he argues, will bring companies down.
- Going to the other extreme of abandoning traditional “heavyweight” management mindset and adopting instead a “just do it” approach, leading to a chaotic management style that draws the startup into a death spiral.
The Lean Startup then aims to be the sliver-lining of these two approaches, with the aspiration to bring a process and a method to the subtle art of running a startup.
Given the high success and popularity of this book, one would expect to see a steep decline in the failure rate of startups. Not quite so. A quick survey on the Internet shows that we are very far from this goal. According to different sources (like here, here and here) > 90% of startups fails. The exact same rate was reported some 6 years ago too. The #1 reason startups fail (what has also been constant over the last 10 years or so), is a lack of market, or inadequate product-market fit — the very same problem The Lean Startup was looking to solve.
What’s at work here? How come following almost a decade of a solid business theory and a set of best-practices, the core pathology that kills startups is still upon us? Might that be some software-based version of “Antennagate”? In other words “ is it possible that we are ‘holding it wrong’”?
I’d like to suggest this is exactly the case, and that despite its popularly, the key lessons of The Lean Startup have been time and again misunderstood, misused and abused, leading to a stagnant low productivity of growth through innovation. Let us count the ways:
#1 — Don’t Sell, Learn
Reid Hoffman has famously said — “If you are not embarrassed by the first version of your product, you’ve launched too late”. It is a controversial statement, but the way I read it is in the understanding that early versions of your products should not be optimized for customer satisfaction but for your own learning.
Ries defines the runway of a startup as the # of pivots it can have before it runs out of cash, but a better measure might be the # of experiments it has, hoping it doesn’t actually run out of cash, but transitions at some point into a delivery organization. Either way, startups would do wise to consider the first stage of their lifetime as a learning phase, where the ‘versions’ of your product are not optimized for customer delivery but for market validation, and given the need to maximize runway, it, by definition, will de-prioritize quality, sleekness, delight and usefulness (unless these properties are in and on themselves part of the learning experience). In that sense, ‘embarrassment’ is actually not a bug, but a feature.
Early-stage companies must realize they are not in the business of selling, but rather in the business of learning.
What happens in practice however is that right from the start, startups are highly focused on building the product and then doubling down on sales to start booking revenues as quickly as they can. Even those adopting a ‘lean’ approach, regard product validation as an inherent outcome of their development process. This approach often proves to be futile, because once you confuse the objective of learning with the objective of delivery, you will find yourself optimizing to two different, and often conflicting, target functions at the same time.
Instead, early-stage companies need to realize that before they have a strong grasp on the market-product fit, they are not in the business of selling, but rather in the business of learning.
#2— Stop Confusing Products with Experiments
To recall, the goal of a lean startup is validated learning and the way of reaching that goal is via a set of experiments. Ries claims that ‘An Experiment is a Product’, but for the first stage of the company, I would actually suggest the reverse of words — ‘A Product is an Experiment’.
What I mean by that is that if what you’re producing is validated learning, then your entire planning needs to be focused on building experiments for the sole sake of learning, rather than building products for the sake of delivery to customers.
This approach has several non-conventional implications, that are rarely represented in the todays’ most startup’s processes and organization:
- Not all ‘outputs’ have to be in the form of a software; some of the best learning experiments Ries provides are actually marketing initiatives or UX mockups with no code behind.
- Your ‘product team’ should include people with a diverge set of skills, such that they can cater to the multi-disciplinary nature of your experiments, including members from marketing, design and even psychology. I wrote about it in the past in more details here.
- Manage your experiments as part of your product roadmap; at early stages, it might be all what your roadmap is comprised of. As the company matures, that ratio between deliverable features and experiments is likely to become more balanced.
#3 — Just Because You’re Agile Doesn’t Mean You’re Lean
Startups like to consider themselves as agile organizations; they change course quickly, deploy rapidly and respond fast to new situations and customers’ requests. These are all indeed key characteristics of a startup, but that doesn’t make them lean.
Lean is not about moving fast, but about learning fast; still, many startups fail to internalize the full cycle of the Build-Measure-Learn loop and focus mostly on the build without spending too many thoughts cycles to the objectives of what they build.
If you’re heading towards a chasm, you might want to slow down a little.
In his book, Ries often advocates against the “let’s ship a product and see what happen”, suggesting that if this is your plan, you are 100% guaranteed to succeed in that that you will see what happens. Without a framework of reference, however, this will give very little value. Building things fast, tagging things as ‘experimental’ as an excuse for low quality, and shipping features with no real feedback loop attached to them may all guarantee you move fast, but they don’t assure your’e moving in the right direction.
And honestly, if you’re heading towards a chasm, you might want to slow down a little.
#4 — Allow Yourself To Be Wrong
Karl Popper, the famous 20th century science philosopher, has coined the term falsifiability to describe the key criteria that makes a theory be considered as scientific. A theory is falsifiable (and hence scientific) if there exists an experiment that can produce results that will make the theory false. (in other words, a theory is considered true, just as long as it withstands that test).
Ries, who draws much of his approach from the corpse of science, supports the same notion as part of his view on testing a hypothesis. If our experiment cannot produce a result that will refute our hypothesis, how would we even know we’re on the right path.
Let’s consider a simple example. Suppose we are looking to add to our product a certain option to automate a key of tasks that otherwise require the manual effort of a user. We are thinking to add a checkbox for users to state their preference; however, we are not sure whether that box should be checked by default.
Suppose that we decide to have it checked, and the majority of the users keep it intact, can we know for sure that it is because they are interested in this feature? maybe they just don’t care (or maybe didn’t even notice it exists). If it’s the latter, that would be a poor signal to follow up on and expand the feature. If, on the other hand, the box is unchecked by default, users will explicitly be required to check the box in, proving a clear signal of validation for the feature. Even a better approach might be to force users to make the explicit decision, by popping up a window they have to go through and set their choice. Consider how in both cases, we are driven by soliciting feedback, and not necessarily by optimizing the user experience or doing “what’s best” for our product goals.
Contrast that with the conventional thinking that is likely to suggest the box should be checked by default, as we want to encourage users to use our new feature. Such approach may improve our (vanity) metrics, but is more of a manipulation than a validated learning exercise. If you don’t allow falsifiability to your design, you can’t expect to get a real feedback.
#5 — Knowing When You Reached ‘Minimum’
Let us now turn now to the last, and possibly most acute, abused practice of The Lean Startup— the Minimum Viable Product. To recall, Ries defines MVP as “that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort”. Sadly, in so many observed cases MVP has taken the shape of “that pretty crappy, but not entirely terrible, implementation that will take the team the least effort to build and will keep the product manager only somewhat frustrated”; in other words, it become synonymous to ‘quick and dirty’.
There’s a popular drawing by Henrik Kniberg that captures that practice
While I agree that the upper side of the picture is a fair depiction of a wrong, but common, interpretation of an MVP, I actually in a disagreement with the bottom half too. The idea behind MVP is not the delivery of fully usable high-quality products to customers, through iterations of an expanding scope, as the picture might suggest. Rather, it is about developing the essential product to support our learning experience and nothing else. It doesn’t need to be complete, fully usable or of high quality. It mostly need not get in the way of what we’re trying to learn. That’s the true meaning of ‘minimum viable’, taking into account the two words that make the equation.
Expanding on our example from before, suppose that the product team contemplate two implementations of the feature:
(1) a single option that set automation as on or off at a system-level.
(2) 5 different sub-options, separated by the type of tasks they act upon, providing together a fine-granular control for the user.
The common application of MVP would likely to be option (1) as it is simpler and will take less time to build. Common, but wrong. If we build the system-level option, and most users choose not to use the feature, how can we tell whether it is because they are not interested , or is it merely because the granularity it provides is too coarse to be usable.
Given that specific hypothesis, option (1) turns out to actually be ‘less than the minimum’ to be considered viable to our learning objective. Minimum must be defined in the context of what we are aiming to validate; sometime it actually requires more effort than what our intuition may tell us.
Wrap Up
Hopefully I managed to put a limelight on some of the key pitfalls startups should avoid; The theory of lean may come across as a straightforward one, but in fact, some of its ideas are more evasive than seems. Paying attention to details is key. Unfortunately, so many startups, including those who speak the lingo, fail to fully internalize the theory to its full impact, resulting in taking the wrong course of action.
Having said that, the common misuse of The Lean Startup is only half of the problem; the other half unfortunately lies within the theory itself. Despite being revolutionary in many ways, it is not without its flaws and shortcoming, In my next post, I will revisit the theory, this time from a more critical point of view.
