World Cup prediction models: why do they go awry?
A case of being precisely wrong over approximately right
As Croatia prepare to take on Morocco in the 3rd place playoff, yet another World Cup prediction has bitten the dust. Why does this happen? Why do predictions go awry? Some reasons are presented.
Prediction for Qatar
The above prediction was made by the University of Oxford. To its credit, it got 2/4 semi-finalists right (50%). This could very nearly have been 3/4 as Croatia edged out Brazil on penalties; as fine a margin as possible. 50% of the quarterfinals were correct but only 1 out of 8 Round of 16 games were correctly predicted (12.5%).
The model is not too shabby. The big question, and laughing point, is Belgium in the Final, and of course no mention of Morocco. In fact, the Atlas Lions were shown to finish bottom of their group!
With both semi-finals played, we know that the model has made a wrong prediction. Other organizations, and super-computers, have also not fared much better. Why does this happen?
Why (and where) models go astray
Past is not a reliable predictor of future
Prediction models do a great job harnessing vast amounts of data. By definition, this is past data. The past is not necessarily a predictor of the future — particularly when it comes to the World Cup.
In fact, the #1 ranked FIFA Team going into a World Cup has never won it.
At Qatar 2022, we saw some shock results. As the level of competitiveness improves, past performances and egos count less and less every time.
The World Cup is a special tournament that brings out unique and unprecedented levels of energy, motivation and transformation, something that no model can capture.
Probabilities are difficult to understand
No one foresaw Saudi Arabia beating Argentina. The Albiceleste, had a 91% win probability. They had 6 shots on target (SOT) and 70% possession, but scored just 1 goal. The Green Falcons had 2 SOT and scored from both.
Prediction models work on probability. A probability is a chance of something happening — not a guarantee. A fair coin tossed 10 times should, in theory, give 5 heads and 5 tails. In reality, it almost never does. For sure, it will even out over the long run. But 5H and 5T over 10 tosses? Most unlikely. The FiveThirtyEight site admits this very succinctly [for their prediction model]: “The 2022 World Cup is only 64 matches, so it’s unlikely that our model will be perfectly calibrated over such a small sample”.
Reality is dynamic
Models are theoretical. Reality is, well, real. This is obvious. We know it — and don’t hold it against predictive models. There is however an aspect of reality that predictive models overlook: dynamism.
Knockouts are zero-sum affairs. Both teams try to win. To do this, they employ strategies, including cheeky innovations.
The 56% — 43% probability for ARG-NED was quite reasonable. What it didn’t account for was the wily tactician Louis van Gaal. 0–2 down, entering the final quarter of the game, van Gaal put on two towering strikers. The Dutch pinged long balls into the mixer, knowing that their 6'2" and 6'6" strikers would outjump the shorter opposing centre-backs. Suddenly we had a thriller on our hands.
Data-driven models cannot account for on-field dynamism and real-time strategizing. Only human knowledge of individual capabilities, assessed and validated over time, can do this.
Underdog effect
Morocco was missing in the prediction. Their run was totally unexpected — wasn’t it? Well, as it turns out, and as illustrated in my earlier post, there is, on average, more chance for an underdog to blaze a trail at a World Cup than not. In 2018 it was Croatia. This time it’s Morocco.
However, which prediction model, especially one from a large institution, will go out on a limb and dare to add an underdog getting into the QF or above? No one. Even if they do, who do they gamble on? The Oxford model (kind-of) chanced its arm on Iran reaching the R16, but got it wrong!
D-I-K-W Pyramid
The above image is quite popular online. It provides a graphic representation of what is also called the DIKW pyramid, where Data, Information, Knowledge and Wisdom are arranged in a hierarchy. Data is at the bottom; wisdom at the apex.
Prediction models do well in the data and information realms. It's the next level up, the knowledge realm, where they start to struggle. This is because here a great mixture happens. Quantitative analysis starts to reach its limit while qualitative and ephemeral aspects start to gain ascendancy.
Take Belgium. The world’s #2 ranked FIFA nation was shown as a finalist in the Oxford model, and others too. Totally logical. Compare this to Youtubers whose predictions I watched. None of them rated Belgium — at all. Safe to say, they did not access or process anywhere near the amount of data these models did. They just knew. People who follow the game have an “eye” that helps them recognize things that data simply cannot.
World Cup predictions are hard. No wonder we see $1 million prizes for getting just group toppers and knockouts right. That’s a mere 32 entries to get right! I don’t think anyone in the world can even get the last 8 right. In Qatar 2022, even the top 4 would’ve been nigh on impossible.
Prediction models rely on data, which allows them to develop a precise picture. Precision however can sometimes detract from accuracy.
Compare this to human (our) predictions. We can’t mentally process the level of data the models have. Many times, we cannot explain our reasoning. The downside of this is that we may have a hard time making fine(r) calls. The upside is that we can make calls like Belgium.
Quantitative (data) models present what looks right.
Qualitative (human) predictions present what feels right.
The strengths of both approaches are also their weaknesses, and vice-versa.
In football, anything can happen. As more and newer teams make credible challenges for trophies, this adage rings truer than ever before. Competitiveness and unexpectedness however make the game beautiful. Perhaps prediction models going awry is not such a bad thing after all.
If you liked this post, you will like my book: FIFA World Cup Finals.
