The Demise Of Nate Silver’s Infallibility and Big Data
Recently I have read many articles here in which authors blethered almost reverentially about ‘Data Science’. Their enthusiasm only tells me they have little understanding of what science actually is. Data science is the latest Silicon Valley buzz phrase of course, a couple of years ago everybody was raving about ‘big data’. Here’s an article I wrote about a year ago about the downfall of big data and algorithms. My next will be on Data Science.
The Demise Of Nate Silver’s Infallibility And Data-Driven Journalism
by Ian Thorpe Jul 18, 2017
With last month’s UK election once again having left national politics totally confused and close to chaos, with both main political parties are in disarray and the minor parties have once again blown their chances of any significant breakthrough by virtue of sheer ineptitude, the big data boys with their graphs, charts and tables, are once again running around like headless chickens because most opinion polls pointed in very different directions yet none came close to predicting the result.
In 2012 some little statistics nerd named Nate Silver was elevated to the pantheon of Technological Gods, when he correctly predicted, having modelled the outcome of the vote on his meta — analysis of opinion polls, that Barack Obama would win a second term as US President. That was the election in which Obama’s opponent Mitt Romney famously threw the fat lady off the stage long before she had even done her warm up scales, when he conceded a key state with less than twenty five percent of the votes counted.
It was the comedian Groucho Marx who said “The show ain’t over til the fat lady sings” in a 1930s comedy. More recently somebody else said, “It ain’t over till it’s over.” With the advent of Big Data and reliance upon data-driven modeling of climates, national economies, political elections, economic performance and sports events we might just as well rephrase it as, “It’s over before it begins.” Which is exactly what one TV pundit did after that Romney concession. When his colleague commented that it seemed a little early to concede, the man, whose name I don’t know and have no interest in rediscovering, said something like, “Come on, you know how the science works.” But did we?
Obviously not, given what happened in Britain’s in / out referendum on continued EU membership and the US election of Donald Trump to the presidency in 2016. On the day before the election the divine wisdom of Nate Silver’s digital models gave both the Keep Britain In the EU campaign and Hillary Clinton better than 80% chances of winning.
Another spectacular Nate Silver cock up in 2016 related to American Football’s top prize Super Bowl, Nate Silver’s FiveThirtyEight.com website predicted the New England Patriots to win. Going into the half-time with Atlanta Falcons leading 28–3, the site gave the Patriots a less than 1 percent chance of winning. FiveThirtyEight tweeted: “That Patriots drive took another 5:07 off the clock and actually dropped their win probability from 1.1% to 0.5%.”
Of course we all know what happened next. In yet another brilliant statistical upset that exposed Nate Silver’s infallible mathematical modelling as no better than guesswork, in the second half the Patriots turned mathematical modelling on its head. They scored 25 unanswered points to win, giving America a timely lesson in how self — belief, determination, and the ability of human beings to rise to the occasion can make mockery of statistics.
It was a lesson for those who tell us we must believe in science and mathematics that sometimes the whole is greater than the sum of the parts by an unmeasurable factor, a reminder of the value of commitment, fortitude and preparedness to risk everything; values that were once an integral part of the American mythos, and drove the European pioneers who founded the American nation, but that increasingly become unfashionable and been sidelined in exchange for the perceived infallibility of data-driven analysis and computer modelling.
Since information (data) replaced wisdom and judgement and became, for the media at least, the highest level of human intellect, as demonstrated by Nate Silver’s elevation to divine status and FiveThirtyEight’s place in pop culture, a cultural shift has taken place with regard to the use of statistics. No longer were the words of nineteenth century statesman George Canning, “I can make statistics tell me anything except the truth,” quoted at those who relied on Big Data, polling, and mathematical models. On TV shows and social media feeds the pronouncements of nerds with tablet computers were treated as if they were inscribed on tablets of stone. Data-driven analysis, whether accurate or not,(and it was not accurate as often as it was) provided a quick and unchallengable way to excoriate opponents in debate who relied on old fashioned things like empirical evidence. “Silver gives Hilary a x++ percent chance to win the election” became the Trump card in any debate on the US election. Nate Silver gives the Remain campaign x+ chance of winning the Brexit referendum was offered to counter any mention of the anti — E.U. mood in the old English industrial areas of the north and midlands, in South Wales and central Scotland.
And in both cases the fat lady was singing a requiem for big data at the top of her range, but only those with the ability to see how the mathematical models were bring manipulated to produce politically expedient answers could see the abyss opening up under the predictions of the big data boys.
We had reached a point where pundits were willing to assign data modelling more value than observable reality. The irrationality and risk-taking inherent in human decision-making were brushed aside, this was the age of nerds. In politics, sport and weather forecasting, statistical models were held up as unassailable predictors.
And in all cases, in 2016, they were wrong. Trump and Brexit, New England Patriots in the US Super Bowl and unfashionable, unrated Leicester City in the English Premier League soccer, and as for climate change predictions, well snow should have been unknown to European and American children by now, the northern ice cap should have disappeared and we were promised 50 million refugees, displaced by rapidly rising sea levels would be wandering the world looking for a new home by 2012 (that last one was particularly stupid because if all the predictions had come true those people could have settled in the sub — tropical paradises of Siberia, Alaska and northern Canada.)
For his part, no matter how certain Silver’s fans might have been of the infallibility of his model, the man himself, more often than not, would hedge when asked a direct question. Like those climate scientists whose opinions trigger predictions of sure and certain destruction of life on earth by half past two tomorrow if we do not abandon coal and oil NOW!, Nate’s own predictions are fenced in qualifiers, possible, probably, is likely to etc. In October 2016, under a headline that read “Clinton Probably Finished Off Trump Last Night,” Silver commented: “I’m not sure I need to tell you this, but Hillary Clinton is probably going to be the next president. It’s just a question of what ‘probably’ means.” He then spent the bulk of the article presenting the statistical case to back up his claim that Clinton would win, but in the end equivocated again, acknowledging that he could be mistaken, that other signs pointed to the possibility of another outcome.
Earlier in the election cycle, when results of the Republican primary, the Michigan Democratic primary, and the general election proved him very wrong, Silver’s excuses involved not moving the goalposts so much as moving the entire effing stadium. He claimed that certain unprecedented events had influenced voters (Wikileaks publishing documents from the DNC ‘hack’? The Democrat candidate being a known liar and abuser of office for personal gain?) which skewed the initial models. Even after the Super Bowl, in an attempt to make light of the situation, he tweeted: “At least the Falcons won the popular vote.” To which a user responded, “Nate, you don’t get to make election jokes.”
Silver also acknowledged that in any prediction based on mathematical modelling of data, subjective best guesses and assumptions must necessarily be factored into the algorithm. When unprecedented things happen those guesses and assumptions cannot easily be factored out. By saying this Silver admits that statistical analysis of data might work for plotting trends but cannot predict outcomes when people are involved and instinct, emotion and individual circumstances come into play. This was best summed up in an article by David Morris, on Silver’s failure to predict Trump’s victory in the Republican primary: “Unlikely events like the Trump nomination are, by their very nature, impossible to predict.”
The models are not ancient oracles, they do not predict the future. At best they are informed guesses on the outcome of events conformed to the established patters of similar events in the past. The trouble with trusting the Oracle, however, is that when events occur, though there may be a superficial similarity, there at many different factors at play.
Nate Silver’s infallibility is therefore not the issue. Everyone get things wrong. Silver made his money and his name as a poker player, not one of those pallid, sweaty, intense characters who spent their lives in smoky, poorly lit back rooms that we see in movies, but a player against computers in online games. This is interesting because against a computer it’s possible to play the percentages and win. In those smoke filled, dimly lit back rooms, emotions, stresses. personal qualities and other things that cannot be factored into a computer program are at work. And then there are the bluffs and the psychology.
Yet despite his being fabulously wrong many times, and despite his admissions of fallibility, people still cling to his as the ultimate argument from authority. “Nate Silver says,” you will hear as confusion reigns ahead of the UK election, with polls showing leads for the conservatives ranging between one and twelve percent (i.e. a hung parliament to a super — majority). This signals a problem for the scientific, control freak culture leaders such as Barack Obama, Tony Blair, David Cameron and Angela Merkel. Science demands simple answers and seeks to provide such answers to complex and often unquantifiable questions. When those questions relate to a globalist, scientific views of how the world should be, we should be doubly diligent in questioning ‘the science’.
It’s not Silver. He’s just a poster — boy for the Church of Sciencology Cult. The irony is that mathematics is not really a science, just as Julius Caesar was proclaimed a god by the mob, so mathematics has been proclaimed a science by an academic mob. In fact it is, in the truest sense of the word, an art; an artifice, something created by humans. Nature does no mathematics, a couple of years ago in a BBC Science documentary on British TV two mathematicians were burbling ecstatically about how the invention of ‘zero’ (by some ancient Assyrian, Babylonian, Egyptian or Hindu philosopher)had changed our understanding of the universe. Well it may have changed the way we do calculations but it does not change the fact that there is no zero in nature.
There may be an absence of something (breathable air for example) but there can never be zero nitrogen, oxygen, carbon dioxide atoms to nature, that would only be significant to a human trying to measure what was (or wasn’t) present. Jason Rhode, in Paste Magazine, wrote a withering critique of Silver, opening with a quotation from Federalist 55: “Nothing can be more fallacious than to found our political calculations on arithmetical principles.” And yet today many seem to believe that Nate Silver is The God Mathematics made flesh, an avatar of a desire for certainty in the unpredictable and unscientific world of human interactions rather than just a statistician. As novelist Terry Pratchett wrote in “The Thief Of Time,” in any battle between order and chaos, chaos will win because it is better organised.
Invoking FiveThirtyEight seemed, before Brexit and Trump, to bestow upon those who quoted the Silverian doctrines an air of both intellectual superiority and mathematical indifference. “Nate Silver predicts…” is akin to saying “Shut up idiot, what do you know? If you don’t know how ‘the science works’ you are not entitled to an opinion. The numbers can’t lie, because science!” But that appeal to Silver (argumentum ad argentum?) is really an appeal to the desire of insecure people for a totally stable and predictable future.
In the past, while a meta — analysis or poll of polls has been reasonably accurate in predicting overall percentages of the vote for different parties, those percentages cannot project the number of seats won in Parliament, because of the way our system works. Similarly, in the USA in 2016, while Hillary Clinton won the popular vote by virtue of huge majorities in states where her opponent had not campaigned because he knew he could not win, Trump campaigned hardest in states hardest hit by globalism, and won the electoral college.
If it had been a poker game, Hillary would have been playing against computers, Trump would have been sweatily shuffling cards in a dimply lit, smoke filled back room.
Ultimately, an unwavering faith in Big Data has become a sort of religious fundamentalism, an attempt, using only belief and awe of the superior being, to impose order on chaos, an almost fundamentalist approach that borders on statistical fanaticism. Unfortunately, just as mathematics is not a science, neither is statistics, which is more akin to the dark arts that the documented and demonstrable proofs of the natural sciences. It’s an attempt to overlook how little we know and how much we imagine we can control.
With a large segment of the population and an even larger segment of academics, media pundits and politicians eager to reduce human interaction to data points, which enables them to imagine they could control human nature, we run the risk of becoming an increasingly technocratic society where people value safety over stimulation, ease over innovation and convenience over adventure. Ironically, if some Christian preacher talks about The Creation, Nate Silvers fans will scream in unison about the undeniable fact of the evolutionary process, thereby only affirming that they don’t have a clue how survival of the fittest works. (Hint, it doesn’t mean those who work out every day at the gym and health club will lead the way to the future.)