The Science of Superforecasting
What Makes Astonishingly Good Decision-makers Tick?
by Christina Cook
Will Trump or Putin host the other for a bilateral visit? Will the FAA let Amazon test its drones? Will the US withdraw from NAFTA? Some people can predict the answers to questions like these to a surprisingly accurate degree. Two Penn Professors recently discussed the science behind it on Wharton Business Radio.
While the back alleys of bad decisions can lead even the savvy astray, good decision-making skills can pave the way to game-changing insights on how to address — or prevent — problems on a global scale. The decision-making process is a complex one, and the different approaches two Penn Integrates Knowledge professors take to gain a better understanding of it inform each other in interesting ways.
I. George Heyman University Professor Barbara Mellers, who holds appointments in the School of Arts & Sciences and Wharton, has developed decision-making methods that are currently being adapted to enhance the accuracy, precision, and timeliness of intelligence forecasts.
James S. Riepe University Professor Michael Platt, who holds appointments in the Perelman School of Medicine, the School of Arts & Sciences, and Wharton, has helped change the standard approach of brain research to one which deepens our comprehension of how the brain makes decisions.
2016 Election Predictions: The Wrong Side of Maybe
Political forecasters predicted that Hilary Clinton would win the 2016 U.S. presidential election. Donald Trump’s win doesn’t make their predictions wrong.
“There are only two ways to get a forecast wrong,” says Mellers, “if you say an event has a zero probability and it happens or if you say an event is certain to happen and it doesn’t.”
It would be more accurate to say that the forecasters were on the wrong side of maybe. The best of them gave Clinton a 66 percent chance of winning, which means that if counterfactual trials were run in history, Trump would be expected to win 34 percent of the time. Beyond that, Mellers points out, Clinton did win the popular vote, which is what people are more likely thinking when making their predictions.
Emotions in Decision-making: Benefit or Bane?
The unprecedented degree to which emotions dictated voter behavior in the recent election raises the question of whether decisions are made in the head or in the heart. Mellers’ psychological research and Platt’s neuroscience and evolutionary biology-based research have yielded intriguing — and intriguingly different — insights relative to this issue.
Mellers’ studies show that the most accurate forecasters — dubbed ‘superforecasters’ — take a highly analytical, rational approach to making predictions and are largely able to leave their personal feelings out of the equation. From Platt’s perspective, however, emotions have played an important evolutionary role in the brain’s capacity to make good decisions. He suggests that the superforecasters’ unprecedented accuracy may indicate an exceptional ability to integrate their emotional intelligence and mental acuity when making decisions.
Teamwork: The Power of Aggregate Forecasts
Another aspect of the superforecasters’ high success rate lies in their ability to work well in teams.
“When you put the high-accuracy folks together in their own teams, you get this surge of accuracy that goes well beyond what you would expect,” says Mellers.
This finding coincides with Platt’s research, which seeks to provide a more accurate and biologically valid understanding of decision-making in social contexts. His work has yielded novel insights into the ways corporations and other organizations could assemble more effective teams.
The Neuroscience of Superforecasting
Mellers says superforecasting teams excel at finding good — even esoteric — information, sharing it with each other, dividing the labor, and coming up with aggregate predictions that turn out to be not only far more accurate than the general population, but even better than the predictions of intelligence analysts with access to classified information. This has led her to believe that there might be an underlying forecasting skill, akin to a person’s IQ or personality traits.
As a neuroscientist, Platt is intrigued. “I’d love to get a look at their brains,” he says, surmising that the overlapping brain circuitry responsible for social interaction, creativity, and open-mindedness is particularly robust in the brains of superforecasters.
Hybrid Forecasting: The Role of Big Data
We live in the big data world — and now we can identify people who are adept at harnessing this data to make highly accurate predictions of how geopolitical situations will play out. Mellers and her team are working on a new project that combines the two. “Hybrid forecasting,” as they’re calling it, promises to improve prediction accuracy over and beyond what superforecasters and machine data can do independent of each other.
It’s news that bodes well for 2020 Election forecasting.
Christina Cook is a staff writer at Penn’s Office of University Communications
Photos by Scott Spitzer
Audio courtesy of Wharton Business Radio