Panda, Pig, and Pachyderm Predictions

excerpted from Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics

Here we go again.

Panda, Pigs, and Pachyderms are predicting World Cup soccer games.

The last time around, the 2010 championship game pitted Netherlands against Spain, two extraordinarily talented teams from underachieving nations. Which country would finally win the World Cup? I loved the Dutch, who had won all six of their World Cup games, scoring 12 goals while allowing only 5, and had knocked out mighty Brazil and Uruguay. But then I heard about Paul the octopus, who had correctly predicted the winners of seven World Cup games. Paul the Oracle picked Spain, and the world now seemed confident of a Spanish victory.

How could a slimy, pea-brained invertebrate know more about soccer than I did? I laughed and waited for Paul the Omniscient to get his comeuppance. Except he didn’t. The Dutch did not play with their usual creativity and flair. In a brutal, cynical match, with fourteen yellow cards—nine given to the dirty Dutchmen—Spain scored the winning goal with four minutes left in the game. How could an octopus living in a tank have predicted any of this?

Had Paul ever seen a soccer game? Did Paul even have a brain?

It turns out that octopuses are among the most intelligent invertebrates, but that isn’t saying much—sort of like being the world’s tallest midget. Still, Paul made eight World Cup predictions and got every single one right. Not only that, Paul made six predictions during the 2008 European Football Championships and got four right. Overall, that’s twelve out of fourteen correct, which in the eyes of many would be considered statistical proof of Paul’s psychic abilities.

He was truly Paul the Psychic Octopus!

And yet, something didn’t seem quite right. Is it really possible for an octopus to predict the future? Paul’s performance raises several issues that are endemic in statistical studies. Paul was not a psychic (surprise, surprise), but he is a warning of things to watch out for the next time you hear some fanciful claim.

First, let’s look at how Paul made his predictions. At feeding time, he was shown two clear plastic boxes with the national flags of the opposing teams glued to the front of the boxes. The boxes contained identical yummy treats, such as a mussel or oyster. Whichever box Paul opened first was the predicted winner.

Octopuses don’t know much about soccer, but they do have excellent eyesight and good memories. One time, an octopus at the New England Aquarium decided he didn’t like a volunteer and shot salt water at her whenever he saw her. She left the aquarium to go to college, but when she returned months later, the octopus remembered her and immediately drenched her with salt water again.

Paul the Psychic Octopus was living in an aquarium in Germany and, except for the Spain-Netherlands World Cup final, Paul only predicted games involving Germany. In eleven of the thirteen games involving Germany, Paul picked Germany—and Germany won nine of these eleven games. Was Paul picking Germany because he had analyzed the opponents carefully or because he had an affinity for the German flag? Paul was no doubt color blind, octopuses do recognize brightness and are attracted to horizontal shapes. Germany’s flag has three vivid horizontal stripes, as do the flags of Serbia and Spain, the only other countries Paul selected. Indeed, the Spanish and German flags are pretty similar, which may explain why Paul picked Spain over Germany in one of the two matches they played and picked Spain over the Netherlands in the World Cup final. The only game in which Paul did not choose the German or Spanish flag was a match between Serbia and Germany.

Paul wasn’t picking the best soccer team. He was choosing his favorite flag. Paul the Omniscient was just a pea-brained octopus after all.

Another problem is that too many people with too much time on their hands try stupid pet tricks, using animals to predict sports, lottery, and stock market winners. Some will inevitably succeed, and who do you think gets reported, the octopus who picked winners or the ostrich who didn’t?

How many birds, bees, and beasts try to predict something and go unreported because they failed? We don’t know, and that’s precisely the point. If hundreds of pets are forced to make pointless predictions, we will be misled by the successful ones that get reported because we don’t take into account the hundreds of unsuccessful pets that were not reported.

Another problem is that people seeking fifteen minutes of fame are tempted to fudge the data to attract attention.

After Paul the octopus received worldwide attention, a previously obscure Singapore fortune teller reported that his assistant, Mani the parakeet, had correctly predicted all four winners of the World Cup quarterfinal matches. Mani was given worldwide publicity, and then predicted that Uruguay would beat Netherlands and that Spain would beat Germany in the semifinals, with Spain defeating Uruguay in the championship game. After Netherlands defeated Uruguay, Mani changed his finals prediction, choosing Netherlands, which turned out to be incorrect. Nonetheless, the number of customers visiting this fortune teller’s shop increased from ten a day to ten an hour—which makes you wonder whether the owner’s motives were purely sporting and whether his initial reports of Mani’s quarterfinal predictions were accurate.

Paul the Octopus is dead, but one thing you can be certain of at this World Cup is that there will be no shortage of prognosticating pets. Laugh, but don’t take them seriously.