A Dangerous Lack of Skepticism
Last weekend, Slate announced the use of social scientific tools similar to those used by campaigns themselves to anticipate results over the course of the day. Slate rejects, in editor-in-chief Julia Turner’s words, the “paternalistic” stance of the traditional media embargo on publishing results during Election Day.
Slate is making a bold move by ignoring the embargo, but in doing so they also appear to be ignoring the flaws of data science and a sacrosanct principle of both social science and journalism: skepticism.
Both Turner and former Slate contributor Sasha Issenberg lay out the rationale for ignoring the media embargo. It’s based on a precedent set in 1980. In that year’s election, NBC called the election for Reagan at 5:15 Pacific time. The early call opened the possibility that NBC affected West Coast turnout, a possibility Issenberg sets aside based on subsequent social scientific research that “yielded unconvincing evidence that early calls had had an impact.” He then goes on to cite social science research demonstrating increased turnout when people find out that others are voting. As further proof, he points to the two-tenth difference between Obama’s 2012 vote share in Hamilton County, Ohio, and that which Obama’s campaign had predicted.
But Issenberg, an expert journalist of social science, should know better. Extrapolating from a single case and a half dozen journal articles provides support for the idea that voting might not be affected, but it is not ironclad proof. In my introductory statistics class, the most important lesson I teach students is to model uncertainty. No statistical estimate, whether it measures the outcome of the 1980 election, current support for Hillary Clinton, or the link between smoking and lung cancer, comes without some uncertainty. We should always be skeptical that we have found the right answer.
If Issenberg seems to have forgotten this basic principle, it is potentially because he stands to profit from projecting certainty. He is chief strategist for the company, Votecastr, with which Slate is partnering to provide this coverage.
This does not make Issenberg wrong or Slate irresponsible. There is nothing wrong with advocating one’s positions. But I want to believe that most journalists would be skeptical when reporting the claims of someone with vested interests. Issenberg himself decries journalists’ reliance on campaign flaks for news on Election Day as one reason to upend the system because they spin self-serving stories.
Coincidentally, Slate’s announcement came out the same week as a book did warning of the downsides of data science written by another one of its contributors. In Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, Slate Money contributor Cathy O’Neil (a.k.a. mathbabe) argues that algorithmic predictions can — and often do — end up affecting the world in unpredicted ways. She demonstrates in a dozen examples in the way that algorithmic coding ended up influencing the world in ways that perpetuated inequality: teacher evaluations based on small samples of students, college grads denied jobs because of past medical problems, and racial minorities denied loans based on data that proxied race.
The closest analogy to what could happen with reporting during Election Day might be that of the housing crisis. Homes were a historically safe investment. Banks continued to invest in mortgage-backed securities — and the collateralized debt obligations based on them — because no evidence existed of massive mortgage defaults. After 2008, we saw that just because massive mortgage defaults had not happened, it was clearly possible that they could. And, in fact, the efforts of the banks and real-estate industry made the problem worse by creating the conditions for defaults based on their bad models. The biggest lesson of O’Neil’s book is that we should be skeptical of data science because it can lead to bad, undemocratic outcomes.
Just because early reports have not previously affected elections does not mean that Slate’s efforts won’t in the future. The 1980 election was not particularly close; the Iran hostage crisis, stagflation, and disillusionment with Carter’s presidency made it highly likely that Reagan would win. I’m skeptical that Slate could be so sure that their reporting wouldn’t influence 537 voters in a single state, the number of voters that separated George Bush from Al Gore in Florida after the 2000 election.
There are good reasons to challenge existing orthodoxies. Good science and good journalism are based on that very idea. The paternalism of the national news outlets should not be taken for granted just because it has been taken for granted for so long. Slate’s decision to challenge it might, and likely will, provide benefits to political journalism. I know I, for one, will have a difficult time pulling myself away from Slate’s reporting using Votecastr on November 8.
But science and journalism are also both built on the foundation of skepticism. Without it, we end up being overconfident, trading paternalism for adolescent hubris. Just as mortgage default rates were never very high until they were, projections might not affect elections until they do.
Shouldn’t we be skeptical of that possibility for our democracy?
