What is the source of this story and what do I know about it?

The Sniff Test, Question 1

Ben Bayer
9 min readDec 5, 2016

There’s a song in Fiddler on the Roof called “The Rumor” that illustrates the danger of gossip, even when it is undertaken without purposeful deception. Yente starts the rumor with the following stanza:

Remember Perchik, with all his strange ideas?
Remember Tzeitel’s wedding?
Where Tevye danced with Golde
Well, I just heard
That Tevye’s been arrested
And Golde’s gone to Kiev

This gets passed from person to person, and by the end of the song Avram is singing quite a different story:

Remember Perchik
Who started all the trouble
Well, I just heard from someone who should know
That Golde’s been arrested
And Hodel’s gone to Kiev
Motel studies dancing
And Tevye’s acting strange
Shprintze has the measels
And Bielke has the mumps
(Yente)
And that’s what comes from men and women dancing!

Yente is the source of this rumor and she doesn’t even realize it at the end, it’s gotten so garbled (and even she is getting it wrong in the first place). It’s bad enough that a message can get so corrupted just from errors of transcription. Anyone who’s played a game of “telephone” knows how that works. Imagine what can happen when dishonesty is added to the mix.

In fact, you don’t have to work to hard to imagine it: countless people believe fabricated messages they get from complete strangers every day on the Internet. These people are not asking the first question of the sniff test for news: what is the source, and what do I know about it?

Maybe trust him about the weather outside. (Courtesy Wikimedia Commons.)

Why would you believe a story about how Donald Trump touched Rupaul inappropriately in the 1990s, if you’ve never heard of World News Daily Report? Even if you have heard of a source, there’s still the separate question of whether it is a reliable source. But if you haven’t even heard of the source before, why believe a thing it says, especially on a charged topic like this? People don’t come pre-loaded with credibility by default. Maybe we can trust most people to report what the weather is like outside, or where the closest gas station is. But not just anyone is in a good position to score an interview with Rupaul and get him to divulge his deepest, most embarrassing secrets.

Why would you believe a story about a murdered FBI agent involved in investigating Hillary Clinton’s email, if you’ve never heard of the Denver Guardian? Here at least you have the excuse, perhaps, of never having lived in Denver, and not knowing that the main news daily there is the Denver Post. But the name of the city isn’t exactly a trademark, and so anyone can use the name in their web site. And just about anyone these days can set up a web site for spreading fake stories made to look vaguely like a newspaper. As, in this case, they did.

The Internet allows misinformation to be spread at lightning speed. But it also allows you to check that information just as quickly.

You don’t need to know much about journalism to avoid sharing the gossip mentioned above.

First, start by actually reading the story rather than sharing it simply because you like the headline. Does the story sound plausible? Is it reported like a regular news story? With just a few more clicks, you can go to the home page of World News Daily Report to see if they have a disclaimer tucked away about how they are a satire site, or if they are touting other stories that sound ridiculous (like “GERMAN SCIENTISTS PROVE THERE IS LIFE AFTER DEATH”). You might visit the home page of the Denver Guardian to see if they have anything resembling legitimate local Colorado news, and not just a total of three generic placeholder stories that could be true of any locale.

Leaving their site, you can do a Google search to see if anyone (like Snopes) has analyzed the reliability of these sites. You can Google key names and phrases from the article to see if other sources have independently reported the story. But if the only other sites mentioning the story get it from the same obscure source, it’s a good bet that they’re also just victims of unfounded rumor or hoax.

Spreading rumors was bad enough in the days when there was no way to verify their source. It’s even worse in an age where you can learn who the source was and whether it’s a reliable source. The Internet allows misinformation to be spread at lightning speed. But it also allows you to check that information just as quickly.

How do we know when we’ve found a reliable source? This is a question that is wider than journalism. It’s a philosophical question about why we should trust the testimony of others, about anything. Any time we rely on a friend to report the details of a conversation we didn’t witness, or a bystander to a murder to recount whodunit, or any number of experts (like doctors or scientists or technicians) to give their diagnosis of problems we need to solve, we are relying on testimony.

There’s a lot I disagree with in the philosophy of David Hume, but I think he gets something importantly right about the rational basis for our trust in testimony. Hume wrote:

[T]here is no species of reasoning more common, more useful, and even necessary to human life, than that which is derived from the testimony of men, and the reports of eye-witnesses and spectators. […] [O]ur assurance in any argument of this kind is derived from no other principle than our observation of the veracity of human testimony, and of the usual conformity of facts to the reports of witnesses. […] Were not the memory tenacious to a certain degree, had not men commonly an inclination to truth and a principle of probity; were they not sensible to shame, when detected in a falsehood: Were not these, I say, discovered by experience to be qualities, inherent in human nature, we should never repose the least confidence in human testimony. A man delirious, or noted for falsehood and villany, has no manner of authority with us.

In other words, we have reason to trust other people’s testimony when we know they have a track record of accurately reporting what they witness and not lying about it. We can trust others rationally, without having to accept what they say as a matter of blind faith. We can have evidence that people are good at reporting the facts. To acquire this evidence, we have to start with testimony whose veracity we can check directly ourselves. We build from there to acquire trust in the testimony of those whose reports we can’t check directly ourselves. If we couldn’t do this, we wouldn’t be able to know about the distant past, about current events in distant lands, or any number of questions about which we ourselves have little if any expertise.

Reliable if biased. (Courtesy Wikimedia Commons.)

So how does this apply to figuring out which news sources are reliable? Honestly this is a difficult question. Nobody, to my knowledge, keeps anything like an exhaustive account of the track records of different news sources. Even if someone did, there would be a further question of why we should trust them. I’ve been following the news pretty closely for over 25 years. My experience began with traditional print news, expanded with the introduction of online versions of the same, and was then challenged by encountering various alternative online sources sometime after 2001. There’s a longer story here, but in a nutshell, I’ve pretty much come full circle to embrace what is usually derisively referred to as the “mainstream media.”

Trained reporters know how to ask questions and confirm allegations in ways that not just everyone with a blog knows how to do.

My current view is that, with rare exceptions, unless a story ultimately originates in the work of an established journalistic institution (whether liberal or conservative), it’s very likely that the story will fail the sniff test. This claim will of course meet with resistance by many on the right who accuse the mainstream media of liberal bias. Interestingly, hard left critics (like Chomsky) usually accuse the very same sources of being stooges of corporate capitalism. I have the strange experience, sometimes, of reading posts online by my right-wing friends lambasting the left-wing Washington Post, and then hearing my left-wing academic colleagues trashing the Post for being right-wing.

Conflicting accusations of bias like these help demonstrate why the mainstream media is where we should place most of our trust. It’s true that the journalistic establishment is predominantly left-liberal. It’s also true that its reporting is subject to market incentives. Both of these factors leave reporters and editors subject to various forms of bias. But both also imply significant assets. The same factor that accounts for left-liberal bias (journalistic training) also brings with it a certain set of skills that contributes to reliability. Trained reporters know how to ask questions and confirm allegations in ways that not just everyone with a blog knows how to do. The same factor that accounts for corporate bias (market incentives) also brings with it a factor crucial to assessing the quality of anything we consume that is produced by others: accountability.

Consuming reliable media sources should be an exercise in the same caveat emptor logic that we apply to other markets.

The accountability factor is especially crucial: the free press in a free market lives or dies by the reliability of its reporting. If it acquires a reputation for inaccuracy or dishonesty, it loses readers and customers. Its competitors are more than willing to sell papers instead. It is no accident that the majority of “fake news” sites are fly-by-night. Anyone can set up a web site and call it the Denver Guardian, make a quick profit from advertising revenue attached to a phony story, but in a free market with a free press, this ruse will not last for long.

Consuming reliable media sources should be an exercise in the same caveat emptor logic that we apply to other markets. Even if you know that McDonald’s hamburgers aren’t A1 quality all the time, you have far more reason to trust they won’t kill you than you do a random guy selling ground beef off the back of his truck. So it is frustrating that people who would never buy the fly-by-night beef still buy into stories they hear that are roughly of the same shoddy pedigree.

Of course it’s true that left- and right-wing media bias exist. But it’s one thing to be a biased interpreter of the facts, and another to be an unreliable reporter of the facts. The more you learn to distinguish the facts from the interpretation of the facts, the more you can abstract away from a story’s bias to get a grip on the underlying facts it reports. It’s for this reason I’m confident that, in spite of its bias, the mainstream media is the best source of information we have today.

If you’re new to the world of news, you might simply not be in a position to know which sources are reliable and which aren’t. Keep reading. You’ll figure it out. Just try not to accept stories uncritically in the meantime.

If, in the meantime, you are still beguiled by some story and want to share it, I have a passage from a famous philosophy essay to share with you. William K. Clifford was a 19th century English philosopher, who thought we had an obligation to base our beliefs on evidence. In the middle of his 1877 essay, “The Ethics of Belief,” he entertained a possible objection:

“But,” says one, “I am a busy man; I have no time for the long course of study which would be necessary to make me in any degree a competent judge of certain questions, or even able to understand the nature of the arguments.”

Then he should have no time to believe.

Read about Question 2 of the Sniff Test.

--

--