What I’m reading — The Signal and the Noise
You’ve heard of Nate Silver. He’s the accountant-turned-statistician who correctly predicted the 2012 Presidential election results in all 50 states. Nate’s known as the world’s foremost forecaster for this feat and others in the fields of politics and sports, but he’s a good writer too, and his book The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t does a great job of explaining why we are collectively so shitty at making predictions about everything from terrorist attacks to tomorrow’s weather.
Each chapter of The Signal and the Noise looks at a different application of forecasting. The sections on stock picking, sports betting, poker, and computer chess were my favorites. But the book’s real value lies in the author’s explanation of his forecasting technique. Rather than using a frequentist approach, which blindly relies on sample statistics, Nate takes a Bayesian approach, which can be highly subjective, but turns out to work well for those of us who like to make accurate predictions.
Bayesian forecasting is simple and intuitive:
- Form an opinion about the likelihood of something happening, and state it as a probability. This opinion may be based on statistics and will be expressed as a number, but it’s a subjective personal opinion at its core (e.g. “The weatherman said there’s a 20% chance of rain and I trust the weatherman, so I believe there’s a 20% chance of rain”).
- Revise that opinion as new, relevant information surfaces. This is where the money’s at. You’ll ultimately distill another probability, this time based on your interpretation of available relevant information (e.g. you walk outside and observe there are few clouds in the sky — “the chance of rain must be lower than 20%”).
… and that’s literally it.
The math for arriving at a Bayesian probability for our revised opinion looks more complicated, but it’s pretty simple too — nothing more than elementary-school algebra. I prefer the following formula to the one Nate uses in his book because it identifies the individual variables more clearly:
Our revised opinion, the P(H1|E) term on the left we are trying to find, is a conditional probability: it depends on the occurrence of some other event, E. It reads, “the probability of our hypothesis, H1, given event E occurs.” In our example, this is the likelihood that it rains (our hypothesis) given we observe there are no clouds in the sky (our event). To calculate this probability, we:
- Find the conditional probability event E occurs given our hypothesis is true, and multiply it by the prior probability our hypothesis was true (the P(E|H1)P(H1) term in our numerator and denominator). In our example, this is the probability there are few clouds given it’s raining, which we can imagine is very small, like 1%, multiplied by the prior probability it will rain, which we assumed to be 20%.
- Find the conditional probability event E occurs given our hypothesis is false, and multiply it by the prior probability our hypothesis was false (the P(E|H2)P(H2) in our denominator). In our example, this is the probability there are few clouds given that it will not rain, which we can imagine might be like 50% (sometimes it’s sunny when it doesn’t rain; sometimes there are clouds), multiplied by the prior probability it won’t rain, which we assumed to be (100% - 20% = ) 80%.
- Perform the quick calculation specified by the formula: (1% * 20%) / [(1% * 20%) + (50% * 80%)]
… And we reach our Bayesian forecast for rain: 0.005, or half a percent.
This approach is hardly novel — it’s been around for literally 200 years — and as Nate demonstrates throughout The Signal and the Noise, it works well. Yet most people fail to take advantage of the Bayesian technique when forecasting. That is to say, most people don’t revise their opinions based on new information. They believe the “20% chance of rain” when it’s sunny.
Anyway, if you read this whole Medium post, you’ll probably enjoy Nate’s book.