How to Make Accurate Predictions

Philip Tetlock (author of Superforecasting) started the Good Judgment project. It asked people to predict global events where they had very little background knowledge.

The following checklist improved prediction accuracy by 10%:

1. Unpack the question into components.
2. Distinguish as sharply as you can between the known and unknown.
   You can’t know everything. That’s OK. Make educated guesses on the unknowns.
   Leave no assumptions unscrutinized.
   The biggest problem with an analytical approach to problem solving (vs an experimental one) is that you’re particularly vulnerable to have 1 missed variable that messes everything up.
3. Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena.
   For example, if you’re trying to predict the chances an employee is going to retain, start with the average turnover rate.
4. Then adopt the inside view that plays up the uniqueness of the problem.
   Think about all the factors in this particular situation that would cause you to increase the base rate. Similarly, think about all the factor that would decrease it. Weigh them appropriately.
5. Also explore the similarities and differences between your views and those of others — and pay special attention to prediction markets and other methods of extracting wisdom from crowds.
   You can (partially) simulate this yourself by writing down your thoughts and coming back to them a few days later. Another great way is rephrasing the question (e.g. instead of what are the chances she’ll accept my offer to what are the chances she’ll reject my offer).
6. Synthesize all these different views into a single vision as acute as that of a dragonfly.
7. Finally, express your judgment as precisely as you can.
   The end result has to be binary — you were clearly either right or wrong. Avoid language like “significantly increase” — what does significantly mean? Avoid nebulous time horizons.
   Use a finely grained scale of probability.
   People that made predictions to 1% granularity generally fared better than people that made predictions to 5% or 10% granularity.