Thoughts on Probability and Statistics
Wealth, Talent, and Luck: A Misleading Story, Solved By Good Statistics
MIT Tech Review Shares a Misleading Article
The MIT Technological Review have been sharing this article for some time now: If you’re so smart, why aren’t you rich? Turns out it’s just chance.
It wasn’t clear what the intended meaning was, and many people interpreted it as claiming that luck determines wealth, not talent or hard work.
The most successful people are not the most talented, just the luckiest…
With the way the article was worded, I thought this was a fair interpretation. However, the research cited in the article does not say this.
This is what the researchers actually claim.
Talented people are more likely to be successful.
“It is true that, as one could expect, talented people are more likely to become rich, famous or important during their life with respect to poorly equipped ones.”
“… from the micro point of view… a talented individual has a greater a priori probability to reach a high level of success than a moderately gifted one…”
Successful people are more likely to be of average talent.
“…the most successful agents are almost never the most talented ones, but those around the average of the Gaussian talent distribution…”
“…from the macro point of view of the entire society, the probability to find moderately gifted individuals at the top levels of success is greater than that of finding there very talented ones, because moderately gifted people are much more numerous…”
These claims, without elaboration, seem contradictory at first, and could certainly be used to push opposing narratives. We can use this to show how probability and statistics are often counter-intuitive.
Statistics Are Counter-Intuitive
You can “see” a pattern and the opposite pattern when viewing the same data in different ways. Of course, the patterns are only in your mind.
As a simplified example, I constructed a scenario with an exact number of people with average/extreme levels of wealth/talent.
Now imagine that someone (usually a journalist or some non-statistician) conducts a study to understand the wealthy. They look at the 100 people with extreme wealth and notice that the vast majority of them (95%) only have average talent. They then conclude that the connection between wealth and talent is weak, and that talent isn’t necessary for wealth.
However, you can view this from a different angle, using conditional probabilities.
When comparing the probability of someone having extreme wealth, given their level of talent…
…it is clear that the extremely talented have a relatively greater probability of having extreme wealth, but both are low in absolute terms. If you wanted to be extremely wealthy, you would rather have extreme talent than not.
This paints a complete different story compared to before. And if the person from before had bothered to study the people with average wealth, they might have found that
Would they have concluded that extreme talent may help people avoid average wealth, given that most people with average wealth aren’t extremely talented?
Changing The Perspective With Conditional Probability
Notice that the researchers talked about micro and macro points of view. While the viewpoints are different, both give interesting and useful information.
Conditional probabilities are a surprisingly useful aid in interpreting data. Of course, variables should be switched around as much as needed.
For example, you can look for trends within each group. If you are conditioning categorical variables on categorical variables, you can switch the conditioned variable and also explore complements (eg. presence and absence of a trait).
It is not simply a matter of being useful though. Failing to consider conditioning can lead to serious misinterpretations of data. A classic example is Simpson’s paradox, where a negative trend over a full data set can become a positive trend when the trend is considered (perhaps more appropriately) within each sub-group.
So, is wealth related to luck?
More than one might expect, perhaps.
And should people continue to develop their talents?