We Need to Talk About Nate Silver: Media Ethics and Fivethirtyeight
In college, I was a political science major with a focus in statistics. Nate Silver’s 538 was basically a designer drug made just for me. I consumed every bit of his content in 2012, again in 2016, and I suspect if I checked my browser history, it’s my most viewed site in 2019–2020 as well. 538 is my primary source for election news.
A study came out in late February that makes, and backs up, some alarming claims about the impact 538’s “probabilistic horse race reporting” had on the 2016 election, suggesting it may have influenced the outcome. The gist is the way the data is reported isn’t intuitive and it has the effect of diminishing turnout. All polling has some margin of error, it’s usually reported in a plus/minus fashion based on expected vote share. 538 reports it in a probability fashion suggesting, hypothetically, a candidate has an 85% chance of winning, meaning that if the election was run 100 times, a candidate would win it 85 times out of 100. I’ve never particularly liked that way of conceptualizing an event that will only ever occur once, it feels a bit dishonest and obscures the uncertainty of an event into a package that is difficult to digest if you don’t already understand the underlying data. The study is worth the read and I’d recommend anyone interested in this topic to do it. Plenty of think pieces of have been written on that subject, so I won’t rehash it further. However, reading it revived something that has been bothering me about 538’s reporting for a while.
I don’t think what Nate Silver is doing is ethical.
(I use Silver and 538 fairly interchangeably, but I recognize there are other writers in the 538 umbrella. I don’t believe that changes my analysis.)
The problem comes from coupling his well-earned reputation as an objective analyst with his subjective punditry. A lot of this comes from his Twitter account, his podcasts and various articles written by other staff at 538 trading on his reputation. He has significant earned readership, and dollars, due to trust in his data analysis. However, in consuming that, you’re also given a heavy dose of subjective opinion.
The end result is Nate contributes significantly to the media narrative about elections, while also being the thought-leader on electoral objective metrics.
Part of this is perhaps being a victim of his own success. If 538 wasn’t the biggest game in town the effect might not be as large. But I don’t think magnitude of damage ought to be the measure of ethical behavior.
Journalists struggle with this concept heavily. Many make a choice to not vote in elections due to concerns that engaging in the process will color their objectivity. I don’t think this is reasonable, but that isn’t really the point. This is how far some journalists go to try and maintain the image of objectivity.
Walter Cronkite struggled to hold back tears on air when reporting on the Kennedy assassination, concerned with the impact on his reputation.
Dan Rather faced similar concerns regarding his reactions to 9/11, in which none other than Mr. Cronkite came to his defense.
I don’t blame anybody for showing emotion on the air. I don’t think I would trust a reporter, male or female, who didn’t show any emotion.
— Walter Cronkite, 2001
“The ethics of a responsible journalist is to put his or her biases, his or her prejudices aside in an attempt to be really fair to all sides at all times…And my pride is that I think I did that fairly well during my years.”
— Walter Cronkite on the Diane Rehm Show, 2009.
Frankly, I agree with Cronkite that I wouldn’t trust a person who is able to report completely emotionless, the human condition is emotion, but that same human condition is why maintaining journalistic ethics are critically important in order to convey objectivity.
At the risk of accusations of a false equivalence, we are living in a time when these ethical boundaries are flagrantly disregarded by explicitly bad actors. Sean Hannity brags about not being a journalist in defense of open opinion based punditry. Meanwhile, Fox News claims to be an objective reporter of information. Fox is intentionally combining objecting and subjective reporting in a fashion to give an imprimatur of journalistic integrity and authority to opinion pieces and drive a particular narrative as objective based. Fox and Hannity aren’t the only bad actors in this sphere, but they are definitely the most successful at mixing opinion and fact into one.
To be clear, I don’t think Nate Silver is evil. I don’t even think he’s intentionally seeking to influence elections. But I also don’t think intent matters. 538’s reporting seeks to objectively measure concepts like media narrative and “momentum” while being one of, if not the biggest, movers on those concepts. That simply cannot be done ethically. Despite what Hannity may claim, you can’t shift between subjective talk show host and objective journalist by changing your hat. You can’t do it by swapping Twitter handles or being self-reflective about it on podcasts either.
A large culprit of merging interpretive opinion and data is the “forecast”. This is a 538 tool that ostensibly digests all available information in an attempt to inform about results. However, this model is chock full of assumptions and perceptions about the race by our objective analysts. It’s also been all over the place as it adjusts to and digests those perceptions over the last few weeks. It’s a numerical representation of 538’s opinion on the race. To be clear, I’m not suggesting 538 has their finger on the scale, or they are intentionally misreporting data to reach a certain result. But what they are doing is substituting their own judgments for the objective metrics they ostensibly report. It becomes a self-fulfilling prophecy. If Sanders or Biden is the presumptive leader, or Warren or Bloomberg are portrayed as having objective no chance, then it dictates reality.
In being the most trusted source for political polling information, Nate Silver has created a monster of a pulpit. He needs to be more circumspect in how he uses it. When he crowns a front-runner, he advances that narrative and as a result he influences the result. Was Sanders objectively the front-runner? Did that narrative manifest Biden’s resurgence? Did the reporting on Biden’s resurgence overstate the effect and ultimately perpetuate it? We will never know. The voters were told it was happening. It’s particularly curious given that Silver himself acknowledged that Biden performing well in South Carolina was his expected result. Yet, when it occurred, the narrative shift was enormous, and reflected in his “forecast” despite being baked in already.
There is a professional duty when doing objective reporting to report, not create, the news, and Silver defaults on this obligation routinely.
When he opines on the state of a campaign or makes predictions on hypothetical future events, or worse, advocates certain outcomes, he creates those realities while attempting to objectively measure them.
Schrodinger’s Cat posits that in measuring sub-atomic particles that you influence the outcome. Silver has a similar quandary, but feels unbound by the paradox, happily acknowledging whether a candidate is dead or alive, without sufficient respect for the influence he has on the conclusion.¹
Two weeks ago it’s Sanders, now it’s Biden. In a need to perpetuate narrative there will be stories about what will be needed for Biden to secure the nomination, or for Sanders to surge back. That punditry is at odds with the objective measures, as ultimately what is being sold is click traffic.
A critic of my position might say “but Nate doesn’t claim to be anything but a pundit.” I find that false. Nate is an aggressive advocate of numbers in search of truth. It’s admirable. I don’t think he intends to push any narrative other than objective measures. The problem becomes when he tells us, rather than shows us, what that truth is, he becomes less an analyst and more a prophet.
Serious Political Analysis didn’t begin in 2008 when 538 was founded. Polling has been around for decades and scrupulous pollsters go out of their way to maintain neutrality in their product, and despite this continue to have subconscious biases. They carefully vet every word, and the ordering of the questions on their polling scripts for even the smallest perception it might influence the result. In search of what truth is, an ethical pollster takes care not to accidentally decide it themselves. This ethical duty shouldn’t end simply because you’ve decided to be a meta-pollster. Silver decides what polls are credible and which ones aren’t, ranking them on a letter grading scale on the basis on their stability and bias. He is the gatekeeper for objective political analysis. He cannot also be the genesis of the media narrative.
There is clearly a market for Silver’s product, it’s understandable that he wants to sell it. We need to ask ourselves if his product is actually good for society and fair elections. I’m not certain it is. It’s time for Nate to recognize the impact he has on elections and adjust his behavior accordingly.
- It’s been suggested I conflated Schrodinger’s Cat and the Heisenberg Uncertainty Principle. Apologies to all who are now very aware I am not a physicist, especially my high school science teacher.