Success, luck and reversion to the mean
Are you fully acknowledging the role luck is playing in the outcomes? Does a good outcome reflect your good skill, or did you just get lucky?
So you aced the test or got a terrific outcome in that project at work. Pat yourself on the back. You put in the hard hard yards and have been justly rewarded. Maybe you are unhappy with someones performance (a team mate, kid or colleague) and you let them know they have to lift their standards. Thanks to you and your motivating intervention, the outcome next time around is better. Lucky you stepped in or who knows how bad things could have got.
In both examples are you fully acknowledging the role luck is playing in the outcomes? Do you understand the inherent variability that can come from the same process with changing inputs? The outcome of any given event is made up of skill + luck, and depending on the event that outcome can be disproportionately weighted toward either extreme. Poor outcomes can reflect the combination of some skill and a lot of bad luck. Does the fact it is different next time around reflect a change in your process or just a random change in your luck?
Daniel Kahneman highlights that any system that combines skill and luck will revert to the mean over time. When he was asked to offer a formula for the twenty-first century he submitted:
Michael Mauboussin writes in his book “Think Twice — Harnessing the power of Counter intuition” :
Say results are part persistent skill and part transitory luck. Extreme results in any given period, reflecting really good or bad luck, will tend to be less extreme either before or after that period as the contribution of luck is less significant.
This concept is important in predicting the future as
an outcome that is not average will be followed by an outcome that has an expected value closer to the average. While most people recognize the idea of reversion to the mean, they often ignore or misunderstand the concept, leading to a slew of mistakes in their analysis.
Consider the under sampling of failure highlighted in this HBR article by Mauboussin -
The most common method for teaching business management is to find successful businesses, identify their common practices, and recommend that managers imitate them.
If causality were clear, this approach would work. The trouble is that the performance of a company almost always depends on both skill and luck, which means that a given strategy will succeed only part of the time. Some companies using the strategy will succeed; others will fail. So attributing a firm’s success to a specific strategy may be wrong if you sample only the winners. The more important question is; How many of the companies that tried the strategy actually succeeded?
The lesson is clear: When luck plays a part in determining the consequences of your actions — as is often the case in business — you don’t want to study success to identify good strategy but rather study strategy to see whether it consistently led to success.
As with all great bias’s this has a name — “The Mathew Effect”. A model can be set up to show how very small changes in input conditions can lead to a runaway result where one person dominates the results.
The Matthew effect explains how two people can start in nearly the same place and end up worlds apart. In these kinds of systems, initial conditions matter. And as time goes on, they matter more and more. When you realize the magnitude of happenstance and serendipity in your life, you can stop judging yourself on your outcomes and start focusing on your efforts. It’s the only thing you can control.
How do you avoid mistakes associated with reversion to the mean? Here’s a checklist from Mauboussin (Think Twice) that may help you identify important issues:
1. Evaluate the mix of skill and luck in the system that you are analyzing.
- A simple test of whether an activity involves skill: ask if you can lose on purpose… if you can lose on purpose, then skill is involved.
- When something good happens, we tend to think it’s because of skill. When something bad happens, we write it off to chance. So forget about the outcome and concentrate instead on the process.
2. Carefully consider the sample size.
- Daniel Kahneman and Amos Tversky established that people extrapolate unfounded conclusions from small sample sizes.
- The more that luck contributes to the outcomes you observe, the larger the sample you will need to distinguish between skill and luck. Baseball is a good example. Over a 162-game season, chances are good the best teams will rise to the surface. In the short term, however, almost anything can happen. … “In a five-game series, the worst team in baseball will beat the best about 15 percent of the time.” You do not see this in chess or tennis matches, games in which the best player almost always beats the worst, regardless of time frame.
- When a large number of people participate in an activity that is influenced by chance, some of them will succeed by sheer luck. So you have to scrutinize even long, successful track records in fields with lots of participants. Investment track records are a good example.
3. Watch for change within the system or of the system.
- Not all systems remain stable over time, so it’s important to consider how and why the system has changed. One obvious example is individual changes in skill level. An athlete’s age is a good example… above-average athletes revert to the mean over time as a consequence of diminished skill.
- Further, the system itself may change. Stephen Jay Gould analysed why baseball has not seen a player sustain a .400 batting average for a complete season since Ted Williams in 1941. …Gould showed that while the mean batting average in the major leagues has been fairly stable over the years, the standard deviation has shrunk from roughly 32 percent in 1941 to about 27 percent today. The bell of the bell-shaped distribution has a narrower width than it used to.
4. Watch out for the halo effect.
- Much of our thinking about company performance is shaped by the halo effect … when a company is growing and profitable, we tend to infer that it has a brilliant strategy, a visionary CEO, motivated people, and a vibrant culture. Maybe it just got lucky.
Once you’ve embraced the paradox of skill, you’ll see that it’s appropriate to have an attitude of equanimity toward luck. If you’ve done everything you can to put yourself in a position to succeed, you should accept whatever results appear. Some days you’ll be lucky, and the results will exceed your expectations. Some days the results will be disappointing because of bad luck. The best plan will be to pick yourself up, dust yourself off, and get ready to do it again tomorrow
From his book “The Success Equation- Untangling Skill and Luck in Business, Sports, and Investing” — Mauboussin suggest a final important point that the very effort that leads to luck is a skill:
Say that you need to complete ten interviews with prospective employers to receive one job offer. Individuals who seek only five interviews may not get an offer, but those who go through all ten interviews will have an offer in hand by the end of the process. Getting an offer isn’t luck, it’s a matter of effort. Patience, persistence, and resilience are all elements of skill.
Skill is best defined as a process of making decisions. So here’s the distinction between activities in which luck plays a small role and activities in which luck plays a large role: when luck has little influence, a good process will always have a good outcome. When a measure of luck is involved, a good process will have a good outcome but only over time. When skill exerts the greater influence, cause and effect are intimately connected. When luck exerts the greater influence, cause and effect are only loosely linked in the short run.
Let me know what you think? I’d love your feedback. If you haven’t already then sign up for a weekly dose just like this.
More like this from 10x Curiosity
- Pre-Mortem — How to avoid disaster — A simple exercise in looking at what went wrong- before it does.
- The Multi-Armed Bandit — to explore or exploit? — When faced with a decision to go with what you know or strike out in a new direction, which do you choose?
- Rubrics to Standardise how you measure workplace systems and performance — Can a rubric help you set a clear standard?
- Antifragile — Becoming stronger with failure — Antifragile. A thought provoking concept developed by Nassim Taleb in a book by the same name.
- Serendipity and the Adjacent Possible — Why do many great ideas get discovered at almost the same time? How can you cultivate serendipity and move into the adjacent possible with your work?
