The indisputable importance of failure: Book notes on Matthew Syeds “Black Box Thinking”

“This is the paradox of success: it is built upon failure.” (p. 44)

In his book Black Box Thinking, Matthew Syed proposes a learning structure inspired by evolution and natural selection. A process of rapid testing and failing — picking the best at every iteration to enjoy cumulative growth. Plain and square trial and error.

This could highlight one reason why we highly regard people with experience. They have already made the mistakes that led them to success. And often, they share their wisdom so that we do not have to make the same errors. As Eleanor Roosevelt is credited with saying: “Learn from the mistakes of others. You can’t live long enough to make them all yourself.”

But to learn from failure, one must learn to embrace mistakes. Our culture celebrates success, but this shadows the essential struggle necessary to get there. As Syed writes: “Removing failure from innovation is like removing oxygen from failure.” (p. 212)

This book celebrates failure. This book proves why we must fail and the rewards we can reap when we cherish the learnings from our mistakes. In another sense, this book is a great tool for self-confidence: destigmatizing failure and recognizing it as something natural and essential in our complex world.

It also gives learning from failure a sense of urgency. Science has adapted to a mindset of openness and humbleness — knowing that todays cutting-edge theories may well be proven false tomorrow. Similarly, aviation takes failure very seriously and has very rigorous protocols for improving aviation security. However, many other organisations and institutions fail to incorporate a system of open learning and rigorous testing with tight feedback loops to experimentally determine the best way forward. Matthew Syed illuminates the benefits that the criminal justice system, hospitals, public sector and other corporations can reap by following what he calls “Black Box Thinking”.

“Everything we know in aviation, every rule in the rule book, every procedure we have, we know because someone somewhere died… We have purchased at great cost, lessons literally bought with blood that we have to preserve as institutional knowledge and pass on to succeeding generations. We cannot have the moral failure of forgetting these lessons and have to relearn them.” (p. 44)

The book in 3 sentences

1. Learning from failure is essential to all development. Failure shows what went wrong and provide insights into how to become better the next time.
2. To learn from mistakes, one needs a system akin to that of evolutionary natural selection — driven by data, variation, and a natural inclination towards the best survivors. However, one also needs an open and growth-oriented culture. One obstacle is cognitive dissonance — the act where mistakes are ignored or reframed. Therefore, it is important to encourage openness, act early on concerns, and always back up those who speak up.
3. Innovation happens by taking creative leaps to find new ground, and while optimising the current system. The best solution can rarely be planned out beforehand, but those who succeed are those with tight feedback and optimisation loop to quickly iterate over many versions of an idea or product.

Actionable takeaways

1. Expect failures to come. And when they do come, cherish their learnings — knowing they take you one step closer to success.
2. There seems to be two problems at play here: first, learning from mistakes. Second, actually admitting that there have been a mistake at all.
3. Admitting failures is essential for improvement. Pseudo-science cannot improve as it cannot be disproven.
4. Make the administration of drugs a science. Not just the development of new treatments, but also considering how they will be delivered to the end customer. Hospitals made an “alert system” similar to the Toyota factory. Reports could be acted upon. Confusing labels could be fixed etc.
5. Retain hard-won lessons and insights by distilling them to their actionable takeaways, as formalised in checklists, procedures and standards.
6. Creativity is largely a response to a frustration or lack of something in the world.
7. When testing if a solution works, use Random Controlled Tests (RCTs). Try your solution on one group, but let one group remain unaffected. Can you see a difference?
8. Make sure you collect data and set up an environment where you can test your actions. Follow the evolutionary system, leveraging variance and combining a top-down/bottom-up approach.
9. Think after beforehand: The concept of a “pre-mortem”. Before starting, state clearly that the project has failed. The patient has died. We are losing. And then ask for plausible reasons why. This helps consider possible fallacies before they happen, speeding up the process.
10. “Self-handicapping” is doing something that decreases your ability, only to have something to blame. Be aware when this happens and eliminate it.

Favorite quotes

“This is the paradox of success: it is built upon failure.” (p. 44)

“Removing failure from innovation is like removing oxygen from failure.” (p. 212)

“Without a problem, without a failure, without a flaw, without a frustration, innovation has nothing to latch on to.” (p. 210)

“It is very easy to sit back and come up with grand theories about how to change the world. But often our intuitions are wrong. The world is too complex to figure everything out from your armchair. The only way to be sure is to go out and test your ideas and pro-grammes, and to realise that you will often be wrong. But that is not a bad thing. It leads to progress.” (p. 190)

“Instead … crash investigators [distil] the information into its practical essence.” (p. 62)

“Success is about creating the most effective optimisation loop.” (p. 195)

“As Duflo puts it: ‘It is possible to make significant progress against the biggest problem in the world through the accumulation of a set of small steps, each well thought out, carefully tested, and judiciously implemented.” (p. 191)

“Winners require innovation and discipline, the imagination to see the big picture and the focus to perceive the very small. ‘The great task, rarely achieved, is to blend creative intensity with relentless discipline so as to amplify the creativity rather than destroy it; Collins writes. ‘When you marry operating excellence with innovation, you multiply the value of your creativity.” (p. 221)

Bonus

To find the truth, it is often more effective to actively look for the times when the hypothesis breaks down — when it should NOT work — than only seeking the “happy paths”. For example, take the sequence 2,4,6. The question is: what pattern does this match? You are allowed to test any NEW set of three numbers to check you hypothesis. One such may be “even numbers”. So you check 16, 18, 20. It works. You test 100,102,104. It works too. But this is not the rule. Because 5,13,18 works as well. But not 3,2,1 or 5,6,2. In fact, the rule is “ANY sequence of incrementing numbers”. But most fail to find this more general rule by being biased in the beginning.