Lessons From Debunking Efforts in Journalism and Elsewhere
I recently completed a fellowship with the Tow Center for Digital Journalism that saw me study how news organizations handle online rumors and unverified claims. I also examined best practices for debunking online misinformation.
This research is collected in a detailed report that you can download and read for free. It’s called “Lies, Damn Lies and Viral Content: How News Websites Spread (and Debunk) Online Rumors, Unverified Claims and Misinformation.” You can view the data we collected (and continue to collect) by visiting Emergent, our real-time rumor tracker.
Below is a report excerpt that offers a look at current debunking efforts in journalism, and among the skeptic community. If you like what you see, you should sign up for our weekly rumor roundup email.
Find more reads and resources on newsgathering, verification and eyewitness media at FirstDraftNews.com
In the fall of 2013, a story circulated on Facebook about the murder of a Swedish dentist.
Marcia Karlsson, the dentist, was allegedly beaten to death by an ex-boyfriend. In a Facebook post that people in Sweden shared thousands of times, a woman named Madeleine told the story of the murder of Karlsson, whom she identified as her friend. Madeleine said the murder hadn’t received any press coverage because the media in the country didn’t want to write about a crime committed by an immigrant.
It was a story that outraged many people, and Madeleine’s post rocketed around Facebook in Sweden. Karlsson, the murdered dentist, had her own Facebook profile, and she was connected to real people. There were pictures of her and her two adopted children.
Jack Werner, the social media editor for Metro Sweden, saw Madeleine’s Facebook post and wanted to write about it. As he dug deeper into the facts of the case, the story began to fall apart. Court records came up empty. Marcia Karlsson, the beloved dentist, wasn’t listed in any directory. “I found out that the whole story was fake, that the dentist didn’t exist, and that it probably was just a big propaganda thing with a racist agenda,” Werner said.
He wrote about what he found, offering a thorough debunking of the hoax.136 The article could have been a one-off for Werner, but it ended up birthing a regular feature for him and two Metro Sweden colleagues, The Viral Reviewer (Viralgranskaren). Since March of 2013, the trio has worked together to investigate trending stories published by the Swedish press to see if the facts stand up to scrutiny.
“I started thinking about the many fake stories that probably circulated among the some five million Swedish Facebook users, stories that we as journalist[s] would never see because we aren’t friends with all of them,” he said. “I got the idea to do that kind of journalism in a more regular and systemized manner, which also would enable us to ask the readers for tips about what they’re seeing.”
Werner said the reaction from members of the public to the feature “is the best I’ve ever had. I get a feeling that people really wanted and needed this kind of service.” The feature has proven so popular and effective that the reporters were awarded the 2014 Innovator of the Year award at the Swedish Grand Prize for Journalism.
The Viral Reviewer is one of several dedicated debunking efforts that have sprung up at news organizations since 2013. Caitlin Dewey of The Wash- ington Post writes a weekly post, “What was fake on the Internet this week.” It started not long after Gawker launched a similar weekly effort it dubbed “Antiviral.” Adrienne LaFrance originally wrote the column, but when she took a job at The Atlantic it became a dedicated blog. Another site in the Gawker media network is also doing consistent debunking work. Matt Novak writes a regular post about the veracity of viral photos on his Paleofuture blog, which is part of Gizmodo. He and others also contribute to Factually, a Gizmodo blog dedicated to debunking and fact-checking.
Some of these initiatives emerged after Charlie Warzel authored his post for BuzzFeed that predicted “2014 Is The Year Of The Viral Debunk.” Data collected for this paper suggests that the viral aspect of debunking hasn’t yet been realized, but there has definitely been an uptick in dedicated journalistic debunking efforts.
These new endeavors share some similarities with political fact-checking sites such as PolitiFact, FactCheck.org, and The Washington Post’s Fact Checker blog. They are all committed to sorting fact from fiction. But while political fact-checkers focus their efforts on the public statements of politicians, political groups, and public figures, the newer debunking attempts home in on online misinformation and don’t restrict themselves to a particular vertical.
Interviews with journalists engaged in these efforts revealed that, like Werner, they feel this work is the core of what journalism is supposed to be. Still, some expressed feelings of futility or frustration that fakes and hoaxes are so prevalent, and beliefs that news organizations play a significant role in helping them spread.
Members of the skeptic community, some of whom I spoke to during my research, echoed the same sense of frustration. That community includes a diverse group of bloggers, scientific investigators, and others who seek to dispel hoaxes, myths, scams, and other forms of trickery. Their sites include the venerable snopes.com, as well as Hoax-Slayer.com, Doubtful News, and many more.
These skeptics and urban-legend busters also pointed to news organizations as being part of the rumor-mongering problem, as much as they are the solution. Brett Christensen has run Hoax-Slayer.com since 2003, and said he wished the press made more of an effort to debunk falsehoods. “I think it would be a very positive step if the bigger news outlets started to report more on hoaxes and scams,” he said. “Wider coverage might more effectively get through to the Internet public at large.”
Through phone interviews and an online questionnaire, these journalists and skeptic debunkers shared best practices and some cautionary advice about trying to combat online misinformation.
The Value of Journalistic Debunking
The debunking efforts in the press are in some ways an anti-viral viral con- tent strategy, a maneuver to insert themselves into trending content by examining and verifying viral stories. This brings traffic to debunkers, but it also helps spread the truth about a viral story.
One example of a debunking that delivered big traffic and helped provide valuable information came during Hurricane Sandy in 2013. Journalists at The Atlantic collaborated to confirm and debunk viral images that spread in the wake of the storm. Alexis Madrigal, who now works for Fusion, led the initiative. He saw a need because, he said, “So many fake Sandy photos were creeping up, and someone, somewhere, needed to try to sort that shit out.”
Madrigal said it was by far the most popular piece of content he’s ever worked on. Part of the reason for the post’s success was that it provided useful and relevant information at a time of high anxiety. In the language of rumor, it helped people make sense of the situation.
The other reason it was effective, according to Madrigal, is that he paid special attention to the visual aspects of the post. “In my Sandy post, I created a visual language for real and fake images that I think was really effective,” he said. The approach included embedding an indication of the photo’s veracity in the image itself. (It’s similar to other efforts such as PolitiFact’s Truth- o-Meter, which offers a visual verdict.) Here’s an example:
One benefit of approaches like this is that the images travel on social media with their veracity verdict intact. In this case, readers didn’t have to click through to see whether an image was real, fake, or undetermined.
Tom Phillips, who at the time was working for MSNBC UK, was another collaborator on The Atlantic’s post. He had launched a Tumblr, “Is Twitter Wrong?,” that was doing similar debunking work during the storm. Phillips agreed to help The Atlantic. “Part of the reason I did it in the first place was as an experiment to see if it was possible [to debunk in real time],” Phillips said in a previous interview with me. “You know, would that fit into sort of a daily workflow in a newsroom, in that kind of context, or would it just become a massive, massive time suck? And the answer from that was basically yeah, you can do it. You can do an awful lot of it in a way that actually fits in with a daily workflow pretty well.”
Phillips said he subsequently saw more journalists engage in debunking during breaking news stories such as the Sandy Hook shootings and the Boston Bombings. “I saw an awful lot of journalists were actively going out doing debunking,” he said. “Debunking is now part of reporting, basically.”
Caitlin Dewey has written a weekly debunking post for The Washing- ton Post’s Intersect blog since March of 2014. She calls it “the most fun- damental service of all — correcting very basic misconceptions about the world and the news.” Though some viral hoaxes or items of misinformation can be amusing, Dewey said she often encounters “things that are being misconstrued in a way where there is a danger to public knowledge and public safety.”
She believes that her weekly debunking posts fulfill an important journalistic mission — and they get solid traffic. (She declined to share specific metrics.) “They do very well,” she said. “That wasn’t the case initially. I don’t know how long it took them to catch on, but now they consistently do quite well.” After having worked on the column for more than thirty consecutive weeks, Dewey has never had fewer than five items to debunk.
Adrienne LaFrance was also never short of material when she created the Antiviral column for Gawker at the end of 2013. Each week she focused on reporting out the details behind hoaxes rather than simply aggregating what had been debunked. “My goal is always to have reporting behind it because it’s very easy to Google the hoax and aggregate what people have already done,” LaFrance said in an interview before she stopped writing the column. “I’d much rather go deeper. One reason so much gets shared is people not taking time to put out a call or do the emailing back and forth with PR people — and we need to do that.”
Her comment that a debunking is often only a phone call or email away underlines Phillips’ point that this work can be done quickly and without a huge amount of effort. But many newsrooms don’t put the emphasis on asking questions before something is aggregated and pushed out.
Journalistic debunking efforts aren’t only focused on viral images and hoaxes. In Ukraine, StopFake.org operates in English and Ukrainian. It was launched in 2014 in response to the propaganda and misinformation that flowed around the conflict in that country.
A team of volunteers, including teachers and students from the Mohyla School of Journalism, track false claims from Russian officials and their sup- porters. They debunk them in articles and videos. “In our case the most successful thing for us was that no one from the Russian side was expecting us to start debunking things,” said Margo Gontar, a co-founder of the site who works as an anchor for a Ukranian media outlet. “In the beginning there were bold lies which were actuality really easy to debunk.”
Gontar said they initially received upward of 150 emails a day from people suggesting things for them to look into. After just three months of operation the site was attracting 1.5 million visitors a month, according to an article from the Nieman Journalism Lab. Gontar noted, however, that the Russian government’s RT cable channel, which often spreads content that StopFake debunks, has a vastly bigger budget and has announced plans to expand into other languages.
While they are up against a better-funded opposition, Gontar said one mark of StopFake’s success is that it caused the Russian government and its supporters “to change their strategy” and try to create more sophisticated misinformation. In terms of impact, she doesn’t know at what rate Ukrainians and others use the debunking work they do to help spread the truth. “I don’t have stats . . . but I did receive letters from readers who said, ‘You know, I managed to change someone’s mind thanks to your articles,’ ” she said.
But Gontar is also careful to note that their site likely has little or no effect on people whose minds are made up. “People who truly believe that Putin is the savior of the planet and that all Ukrainians are fascists — it’s kind of hard to tell something to them,” she said.
The skeptic movement has its roots in the work of a magician and escape artist. James Randi toured the world as the Amazing Randi, performing remarkable illusions. Randi wanted to entertain, not deceive. When he saw people (like Uri Geller) claim to have special psychokinetic abilities, he challenged them to reproduce their powers in a controlled environment. (Geller, among other things, would bend spoons using what he said was the power of his mind.)
In one famous incident, Randi worked with the staff of The Tonight Show With Johnny Carson to ensure that Geller couldn’t pull any tricks when he appeared. Geller’s ensuing guest spot was a complete failure, and he lived it out in front of millions of viewers. Yet, rather than be exposed as a fraud, Geller’s devastated reaction earned him sympathy from the audience — and an instant booking on The Merv Griffin Show, among others, according to a profile of Randi in The New York Times Magazine. “To an enthusiastically trusting public, [Geller’s] failure only made his gifts seem more real: If he were performing magic tricks, they would surely work every time,” the Times reported.
The article included an explanation of why Randi prefers to call himself a scientific investigator, rather than a debunker:
Because if I were to start out saying, “This is not true, and I’m going to prove it’s not true,” that means I’ve made up my mind in advance. So every project that comes to my attention, I say, “I just don’t know what I’m going to find out.” That may end up — and usually it does end up — as a complete debunking. But I don’t set out to debunk it.
Randi has inspired many people to become scientific investigators. Tim Farley is a software developer by trade, but he also runs websites to help people learn how to judge the veracity of the information they see online. “I feel the best route is to encourage people to become their own fact-checkers and show them how the techniques used by the pros can be used by anyone now that Google exists,” he said.
One of Farley’s websites, WhatsTheHarm, attempts to calculate the harm of misinformation, whether it comes in the form of parents who won’t vac- cinate their children or people who rely on faith healing. As of November 26, 2014 the site listed the total harm at “368,379 people killed, 306,096 injured, and over $2,815,931,000 in economic damages.” These figures are anecdotal, and are based on the work of a group of volunteers who identify examples collected from news reports.
Another man, Robert Todd Carroll, has been engaged in skeptic work for two decades and maintains The Skeptic’s Dictionary. Like Randi, he said he doesn’t approach his work with a debunker’s mindset. “I don’t always begin an investigation with the idea of debunking an idea,” he said. “My initial inquiries into acupuncture and homeopathy, for example, were attempts to understand everything that was known about these practices and what scientific evidence there was for the many claims made about their efficacy. I ended up debunking both because the evidence led me to conclude that both are little more than placebo-based medicine, known to shamans centuries ago.”
Benjamin Radford, also a scientific investigator, echoed this: “Of course often a mystery is debunked when it is explained, but I try to remain open- minded about the subjects.” Radford is deputy editor of Skeptical Inquirer magazine and a research fellow with the Committee for Skeptical Inquiry, a nonprofit educational organization. He has written many books on skep- ticism, myths, and paranormal investigation, including (fittingly) Media Mythmakers: How Journalists, Activists, and Advertisers Mislead Us.
One skeptic who takes a very journalistic approach is Sharon Hill. Her web- site, Doubtful News, is her attempt to utilize SEO, clickable headlines, and other traffic-generating tactics to combat misinformation, dubious claims, and pseudoscience. “I try to get the story out as soon as possible and utilize important keywords in the story and the title,” she said. “Keywords really are key in getting search results. The earlier you can get eyeballs on that story, the more likely your story will come up in a search.”
Hill will also reach out to relevant online communities and aggregators to help her work reach a larger audience. “I attempt to promote the story to interested parties, if it might get attention on Reddit or Fark.com, if I think I have an interesting angle or catchy headline,” she said.