Is the Boeing 737 MAX safe?
Two high-profile crashes in less than six months have raised questions about the safety of the latest iteration of the Boeing 737, the MAX. Aviation authorities in various parts of the world — but not the United States — grounded the aircraft within days of the second crash. But do two fatal crashes prove that the aircraft is unsafe? This question is fundamentally statistical in nature — it is equivalent to asking whether bad luck could explain these two crashes, or whether they indicate some underlying flaw with the aircraft.
Historical crash rates
To understand how improbable these crashes were, one must first appreciate the incredible safety of modern aviation. In over 87 million flights by the MAX’s predecessor, the 737NG, there have been only seven “hull loss” accidents with fatalities. Only five of these involved double- or triple-digit fatalities. The rate of “catastrophic” accidents was thus less than 1 per 17 million flights. The 737’s main competitor, the Airbus A320, sports a roughly similar safety record. The MAX has flown approximately 360,000 flights with two catastrophic accidents, or 1 per 180,000 flights. This rate stands out as almost 100 times higher than its predecessor. With these statistics we can ask, if the MAX had the same safety level as the previous generation 737 or the A320, what is the chance that we would have observed two catastrophic crashes so quickly after its entry to service?
The odds of crashing
There are many approaches to answering this question. All, however, yield the same answer: the MAX does not meet the safety standards set by the 737NG or the A320. The easiest way to see this is to note that, since its entry, the MAX has accounted for 1% of flights by 737NGs and A320s but is responsible for 100% of catastrophic crashes. If it had the same safety level as its contemporaries, the chance that the first catastrophic crash would hit the MAX, instead of a different model 737 or A320, is approximately 1 in 94. Thus the first crash was already cause for concern. But it was not definitive. Among other things, we did not set out to test whether the MAX was safe, as there was no reason ex ante to believe it would be less safe than any other new aircraft model. If Boeing and Airbus were to introduce dozens of new aircraft models over many decades, all of them just as safe as the 737NG and the A320, eventually one would crash as quickly as the MAX did.
The second crash foreclosed this possibility. The chance that both the first and second crashes would hit a MAX, instead of another model, is approximately 1 in 8,800, or 0.01%. By the second crash the MAX was already under intense scrutiny, and regulators were implicitly testing whether it was safe. Mere bad luck is no longer a plausible explanation. Instead the only remaining explanation is that the MAX is, for whatever reasons, less safe than other 737s and A320s.
How likely is it that the MAX is flawed?
It is tempting to conclude that the data reveal a 99.99% chance the MAX is unsafe, but that would be misleading. After all, most new aircraft models safely enter service. With enough new entries a string of improbable bad luck will eventually strike one of them. A formula known as Bayes Theorem allows us to quantify how likely it is that the MAX is unsafe. If we believe that the chance of a new aircraft having a serious flaw is 1 in 10, then Bayes Theorem reveals that the chance that the MAX is unsafe is 99.9%. Alternatively, suppose we have strong faith in Boeing and believe that the chance of a new aircraft having a serious flaw is only 1 in 100. Then Bayes Theorem reveals that the chance that the MAX is unsafe is still 98.5%. Even optimistic observers should thus be concerned about the safety of the MAX.
Statistics and probability reveal that the MAX is unsafe, but they cannot tell us why. Only thorough accident investigations by trained, impartial personnel will reveal these details. Until we know the causes of the two accidents, however, safety-conscious air carriers and authorities that understand data science are likely to continue grounding the MAX.
