An Introduction to Cognitive Biases in Startups

“When we remember we are all mad, the mysteries disappear and life stands explained.” — Mark Twain

We are all affected by cognitive, or unconscious, biases. Worse, we are biased to believe that we don’t suffer from bias, or at least not as much as everyone else does![1] But suffer from it we do: you, me, our customers, our investors, our co-workers, everyone. Really, everyone.

Operating under the effects of a cognitive bias won’t always mean that we make an incorrect decision or that we won’t obtain a desired result. But in the risky, rollercoaster world of startups, where we all want to increase our chances of success, it pays to understand our own thinking and to improve our ability to make objective and informed decisions.

Working in the business of startups, we are especially prone to bias because:

• the industry is particularly fast-moving; we often have to make decisions based on incomplete information; and
• unlike the reassuringly concrete fields of mathematics or hard science, the business end of a company involves the fuzzier worlds of economics and human behaviour: where ambiguity lives, bias thrives.

When we start with a particular point of view, or when we really want something to be true, we have a natural tendency to seek out information that confirms our viewpoint. We prefer to answer the question “why is this so?” than the question “why isn’t this so?”.

When faced with ambiguous information, we choose to interpret it in a way that fits with our preferred theory.

Even faced with unambiguous contradictory information, we don’t tend to change our view. Instead we do two things: (i) we subject the contradictory information to far more scrutiny than we do the confirmatory information and (ii) we try to rationalise why the contradictory information doesn’t apply to us or to our particular situation.

Imagine the following conversation:

Founder 1: We need to show a larger total addressable market. What’s the largest figure we can support?

Founder 2: I dug through a heap of material and found a paper that says our total addressable market is US$800M.

Founder 1: What about those Gartner and Forrester reports said that our total addressable market was worth US$100M?

Founder 2: Those reports only looked at iOS products. Our product is on Android too, so I don’t think Gartner or Forrester are relevant here.

In the above case:

• Founder 1 has set the parameters of the desired result (“…the largest figure…”) even before the research has been started.
• Founder 2 has searched for and zeroed in on supporting information, possibly (probably) focusing only or mainly on information that supports Founder 1’s framed goal.
• When faced with authoritative contradictory information from Gartner and Forrester, Founder 2 does not dismiss that information out of hand but seeks to explain why the information doesn’t apply to his product. In this case, we have no idea whether the fact that the product is on a second platform (Android) is a valid reason to discount the reports, but the mere existence of the second platform, without further context, is not basis enough to ignore expert opinion.

Like gin and tonic, like VPs of sales and their buddies they worked with in the last place, cognitive biases go hand-in-hand with logical fallacies.

Here’s another conversation:

Founder: We increased our marketing spend.

VC: Oh? And how is that going? Did it increase your revenue?

Founder: We thought it might increase revenue, but it turned out that our click-through rates went through the roof so we’re really delighted. Our click-through rates had been well below average the last couple of months.

Let’s look at the two substantive topics of this short conversation.

Revenue vs Click-Through Rate — Subjective Optimisation

Although the founder says that she initially supposed that the increase in marketing budget would lead to an increase in revenue, it seems that she hadn’t declared “revenue increase” to be the specific measure of success. In the absence of an increase in revenue, she is suffering from “subjective optimisation”: moving the goalposts of what is considered “success”, from revenue to click-through rate. Before taking the action, she may not have considered an increase in click-through rate to be the measure of success, but she is now using it to declare herself delighted with the outcome.

Increase in Click-Through Rate — Post Hoc Fallacy and Ignoring Regression to the Mean

Even if we accept that an increase in the click-through rate is indeed a sign of success, we still have problems.

Our founder is making an assumption about causality that may not be true. We do not know if the increased marketing budget was the cause of the increased click-through rate, and so we are potentially falling for my particular favourite logical fallacy, post hoc ergo propter hoc (Latin for “after this, therefore because of this”). We have a tendency to assume that, because something happens after we take a certain action, it must be because of that action. It doesn’t mean that we are automatically wrong in determining the cause, but we need more data to increase the odds that we have correctly identified the cause.

A final error that may have been made is the failure to recognise regression to the mean. For any extreme result (positive or negative), the likelihood is that later results will be less good or less bad than the extreme. If click-through rates were particularly bad (“…well below average…”) in the months before the increase in marketing spend, statistically the results were more likely to be better in the following months, even without any increased marketing spend.

What can we do?

It is neither practical nor particularly enjoyable to apply a rigorous scientific process to every single choice we make. It is also not a winning strategy at cocktail parties.

If a decision is important or strategic, we should take extra care that we, and the people around us, are not succumbing to bias. Ways to combat the effects of bias are:

• Ask someone to put on a “black hat”[2] to argue against a particular proposition.
• Where we find backup material for our opinions, scrutinise it and deliberately spend time looking at contradictory material.
• Set the tone to encourage honestly held alternative points of view.

Not defining in advance what “success” looks like for a given strategy or tactic is one of the mistakes I see most frequently in the startup industry. It stems from our reluctance to be shown to be “wrong” in our opinions and relates to our natural fear of failure. After all, if we don’t define success in advance, we can never fail…

To increase our odds of proving that an assumption is correct, we should embrace the scientific method:

• Take our base assumption — “increased marketing spend will increase click-through rates”.
• Outline our actions — “we will increase our marketing budget by €X this month”.
• Detail our expectations — “we expect click-through rates to increase by Y%” — this is our yardstick for success.
• Be honest about the results.

The battle to bring an idea to fruition and to build a viable business on top of that idea is a tough one. People who undertake that battle need to be optimistic, probably more so than the normal person-on-the-street. But when we desperately want to believe something, our biases can kick in to convince ourselves of things that just aren’t true. There is a fine line between optimism and delusion.

I have outlined only a very small sample of cognitive biases and logical fallacies here — I highly recommend that you take the time to explore and understand these and more. Being biased doesn’t mean that we are bad people; it is part of human nature. It is important not to deny our biases but to recognise them and to resist them when it really matters. Recognising biases and becoming more self-aware does not weaken us; it helps us to optimise our decisions in the ever-changing world of startups.

[1] Pronin, E.; Lin, D. Y.; Ross, L. (2002). “The Bias Blind Spot: Perceptions of Bias in Self Versus Others”. Personality and Social Psychology Bulletin 28 (3): 369–381

[2] See de Bono, E. (1985). Six Thinking Hats: An Essential Approach to Business Management. Little, Brown, & Co. New York.

References:

Thomas Gilovich “How We Know What Isn’t So”

David McRaney “You can Beat Your Brain”

Daniel Kahneman “Thinking, Fast and Slow”

Dan Ariely “Predictably Irrational”