The Problem With Startup Postmortems

And Why You Won’t Learn Anything From Them

Wise Men Say

If there is one thing more popular in startup community than writing “How I did it” posts, it has to be writing startup postmortems. They regularly rise to the top of Hacker News, there are whole websites dedicated to aggregating these posts, and Founder Institute even published a list of lists of them! The post is named “Keep Your Startup from Failing with this Guide”, but can you really?

People love the idea of learning on someone else’s dime. There are famous quotes about learning from other people’s mistakes:

Only a fool learns from his own mistakes. The wise man learns from the mistakes of others — Otto Von Bismarck (supposedly)

or

Those who cannot remember the past are condemned to repeat it — -George Santayana (apparently)

But can we learn from the history, and, in particular, what can we learn from failed startups?

Mistakes Were Made

These are some of the typical excerpts from post-mortems:

We made deadly cultural and strategic mistakes. We tried to build a sales effort too early, with too weak of a product after initial financing and waited too long to address the “nice to have” problem. We didn’t make [startup] self-serve to let anyone come and get it. We didn’t focus on learning & failing fast until it was too late. And didn’t care/focus enough about discovering how to market [startup].

We focused on engineering first and customer development second. Most of the mistakes we made developing our test accelerator and, later, [service] boiled down to one thing: we should have focused more on customer development and finding a minimum viable product (MVP).

And even things like:

Our monthly operating deficits were too high, and even though we continued to get better at acquisition, each small success actually saw our cash curve decline further because our density remained flat.

(I think he’s saying they ran out of money)

What are these writeups telling us, though?

We Failed Because We Did Stuff That We Did — Circular Reasoning Problem

Postmortems are plagued by circular reasoning. Their insights may be right, but what these founders are telling us is what happened, not why it happened. The typical story comes down to some version of we didn’t get the traction; we ran out of money, or we couldn’t get product-market fit, along with the context of that particular startup. These insights are constructed after the outcome is known, so we can never be sure how much these explanations explain, versus merely describe what happened.

So, when can this story telling be instructive for the future even if we not sure about the causality?

In the situations where we encounter many similar circumstances, we can rely on previously experienced events. For example, when firefighters enter a burning building, they can bypass complex cause and effect relationship in assessing the situation, and can rely on their training and practice, because they have been exposed to very similar conditions many times before.

Unfortunately, with startups, history only happens once. Every startup is in some important way different from every other startup that came before. We want to learn lessons from the history, and it’s easy to fool ourselves that we have learned more than we have.

Most people understand that success stories suffer from survivorship bias. We are aware that there are startups that did the same things but nevertheless failed. Somehow, the same reasoning is not applied to analyzing failures.

Post hoc Fallacy

Let’s say the founders have identified five reasons their startup failed. Even if we believe that these are all identified correctly, it is also true that many startups also did those five things but didn’t fail. But when we are analyzing failed startups, we stop looking at all the ones that didn’t fail, because we are not trying to explain successes. We are seeking to explain failures. And we explain them by recalling what we did to get where we are.

This phenomenon is called post hoc fallacy. Post hoc fallacy states:

Since event Y *followed* event X, event Y must have been *caused* by event X.

So, when discussing startups, we have all these potential causes of failure that can seem plausible. And this is why we are tempted to infer a causal relationship when all we have witnessed is a sequence of events. We do this because it feels reassuring to be able to attribute the failure to a handful of clearly defined reasons, but this is just how we would like the world to work, not how it necessarily works. When you are reading a startup post-mortem you are not learning why something happened; you are just reading a particular description of what happened.

Do these writeups even accurately describe what happened? Are we learning all the relevant facts or just some subset of the events that the founder recalls for some reason or other? What we are reading is a story, and the explanations concentrate on what is interesting and salient, giving it special significance and meaning. These accounts make the story coherent and eloquent, but the desire for closure and meaning in the mind of the founder forces the story into simple, linear narrative, glossing over the complexity, randomness and ambiguity. The answer is never: I just don’t know.

If I Only Knew — Learning From Other People’s Cognitive Biases

This drive for meaning is understandable. People need closures and from the psychology research, we know that people judge simple explanations more likely to be accurate than complicated explanations.

What also founders freely acknowledge, is that they were victims of various cognitive fallacies, that they are aware of in hindsight. They were victims of optimism bias, confirmation bias, escalation of commitment, planning fallacy, and many other cognitive errors. Ever since behavioral economics become popular, people will tell you, that as an entrepreneur, you need to watch out for and avoid cognitive biases. But is this possible?

Can you learn to recognize biases in yourself and eliminate them? Short answer: probably not.

Daniel Kahneman is the father of behavioral science. He has worked for the past 45 years on discovering these biases, and uncovering how we think, why we make cognitive errors and what can we do about it. And this is what he says when asked how you can improve your thinking:

People feel good when they are learning about all these ways we make mistakes*, but when people are actually making mistakes, they are so busy making them, that they don’t have time to correct it.

*No wonder “Thinking Fast and Slow” has become perennial NYT bestseller.

Having this in mind, is there any other reason to read startup post-mortems?

Startup Postmortem, What Is It Good For?

Of course, the real reason we are interested in these stories is not that we care that much why a particular startup failed, it’s that we believe in the power of generalization. When we are trying to understand why a particular startup failed, we are really asking why startups fail in general. We are hoping for something that we will be able to apply in the future.

So far I have shown that reading startup postmortems won’t teach you why particular startup failed, and it won’t teach you to reason better about your startup. Therefore, are they useful for anything at all?

They do seem to touch people on a personal level. They document the struggle, and we empathize with the people who have been through it. Maybe it makes us more humane and humble, and maybe that’s the real value of reading them.

Delusional Optimism Redux

And to end with a final note on optimism bias. What Kahneman refers to as delusional optimism is a hallmark of being an entrepreneur. If founders knew the real odds, we’d all be living in caves now. But this wild optimism has advantages for the entrepreneurs as well.

People who are genuinely optimistic raise more money than the people who are realists (who are better at assessing odds) and their infectious optimism gets others to work for them harder. Therefore, your delusional optimism does make it more likely you’ll end up being successful; and it genuinely changes your reality. Being right and rational is not always optimal, as it turns out.