Survivorship Bias

From saving lives in World Wars to choosing the right school, what’s hidden from the eyes makes all the difference

This is not a kid’s bad attempt to colour a plane, no, it is the image of a WW2 bomber returning from a mission, with the red dots representing bullet holes from the enemy shooters.

At first glance, one might think to strengthen areas of the plane which suffered the damage. But, statistician Abraham Wald took survivorship bias into his calculations and reinforced and armoured parts of the plane with little to no damage. He said that the parts with the damage show that they can take hits and that the plane can still return home, but when the bomber got shot at places with no damage, they crashed, leaving no evidence.

His work is considered seminal in operational research and he saved hundreds of lives with this inference.

When one only looks at the ‘survivors’ and makes their decision or strategy, without giving any heed to the ones that ‘failed’ in that particular situation, they fall prey to survivorship bias.

Survivorship bias explains why our grandparents believe that things made nowadays are made of lower quality than 50 years ago because they only see the objects that survived those 50 years and not the huge amount of products that couldn’t.

Survivorship bias creates a falsely optimistic view of something that should perhaps be treated with more caution taking into account every aspect.

Survivorship Bias in

• Business -

Media extensively covers and glorifies all the businesses that earned millions, every young entrepreneur dropout who became a billionaire. Seeing them everywhere, a young college go-er might believe that dropping out and working on a startup will surely lead to their wealth. But this is a very wrong deduction because they do not account for the thousands of failed startups and college dropouts who could not achieve success. In fact, post-graduates have a higher chance of earning better money compared to those without any degree or dropouts.

Hearing successful people defy the odds is encouraging: if they achieved riches without university, it’s easy to believe, then surely I can, too.

But this is another example of survivorship bias,

Perhaps it would be better to look at businesses and startups that did not succeed and learn from their failures to get a clearer idea of running an enterprise. Young entrepreneurs can use this to increase their chances of success and get a realistic idea of the chance of failure.

• Finance -

A mutual fund is a financial vehicle that pools money collected from investors and is managed by a professional money manager, who then invests in things like stocks, bonds, and other assets.

Survivorship bias in finance occurs when studies are published on various mutual fund returns but they do not include those mutual funds which do not exist anymore, only including active mutual funds.

Those mutual funds might have been closed due to mergers or poor performance. For example, if during a recession certain mutual funds close due to inability to adapt, they would not reflect in the studies and would lead to a positively skewed report which is actually far from the reality.

Therefore, if one wants to evaluate mutual fund returns, one must consider all mutual funds that meet the evaluation criteria in the time frame irrespective of whether they exist or not.

If in a scenario a 1000 mutual funds exist, imagine 10% of these funds stop working by the end of the year. If the researchers do studies on the returns of mutual funds at the years-end they would fall prey to survivorship bias and paint a false picture of returns.

This is illustrated next-

Here if we consider the returns of only active funds, it comes out to be +15%. But, if our study considers all the mutual funds that met our criteria, the average returns come out to be only +6% which is three-fifths less than the one calculated with survivorship bias.

• Education -

In India, millions of students give competitive exams like JEE and NEET every year. These are seen as a way to success for many households and understandably parents are keen to find a way to get their kids in front of the line. Knowing fully well about the obsession with such exams, many huge coaching centres have popped up throughout the country, each selling the dream of qualification in these exams.

Huge posters are put up by these institutions and millions are spent in ad campaigns trying to lure parents and kids into enrolling in their programmes.

Here, survivorship bias comes into play.

Suppose an institution, say XYZ, had a student of theirs, A, secure All India Rank 1. Now, XYZ is going to spend huge money in generating hype around this student and having extravagant displays of this achievement.

When the average parent or student sees this, they get attracted to XYZ and think of joining it, with the dream of their kid also achieving something on that scale.

But, they do not consider the success rate of kids in that institution, they miss the number of students who failed the exam. Suppose this student A, was a major exception and most of the students in that institution are extremely disappointed with their result.

But the families fail to research factors like these and fall prey to survivorship bias.

Maybe they should’ve chosen an institution with a more consistent and widespread record rather than an outlier success.

For example -

It can be seen through this table, while XYZ has more number of students with an amazing 99–99.9 percentile, ABC does far better in respect of students getting between 90 to 99.9. Thus, ABC might be a more practical choice for students wanting to increase their chances of a decent rank albeit not the best one, as a lot of students in XYZ fail to even cross 90.

What can we get out of all this?

We should always look for what’s missing in a given data set, and give thought to the possible data points that were on the same path but could not make it to the final analysis.

It’s hard to rationally consider survivorship bias when presented with examples of success, but it is necessary to do so to prevent coming out of research data with a skewed result, and make the correct decisions, whether they be for a business prospect or personal goal.

Understanding survivorship bias is a tool to cut through the noise and see a data set for what it truly represents. So the next time you want to help in a war zone, you know what to look for.