How should journalists talk about polarizing statements by politicians? We tested it.
This post was written by Trusting News research partner Joshua Darr. The research was supported by the Carnegie Corporation of New York and the Reilly Center for Media and Public Affairs at LSU.
When a politician makes a false claim, it puts journalists in a bind — and politicians know it. Journalism should be simultaneously accurate, impartial, enforce accountability, and support democracy and political participation — but when an official repeats a disproven claim about election integrity, it can feel difficult to uphold all of these values at once.
Constituents should know what their officials are saying, but it’s also true that repeating purposeful misinformation can do a disservice to audiences. It’s often not the right answer to report “both sides” of a story in a neutral tone as if they have equal merit. Yet a strong stance that minces no words about false claims that undermine confidence in voting may never reach the ears of the voters who need to hear it.
And there is another factor as well: What do readers actually prefer? What gets read and what doesn’t? We tested it with a survey experiment in the state that has been ground zero for these false claims: Arizona.
We used a technique called a “conjoint experiment” that randomly varies several attributes at once, in this case about news stories. There were several possible Arizona-based outlets, labels, headlines, etc. that could be combined in many ways. These headlines did not actually appear in these outlets; instead, we mixed and matched outlets’ headlines, labels, and author affiliations randomly, using actual logos to make the treatments somewhat more realistic.
Respondents were asked to choose their preferred option from the options that are presented. Note: We based the headlines on a real story, and used the politician’s name in the survey, but will redact it here so that our experimental treatments are not mistaken for real headlines by Google searches. That means respondents saw something like this:
Using this method, we can determine how much each of these individual options makes the biggest impact on readers’ choices. Today I’ll focus on one aspect of these treatments: the headline.
We based these mock headlines on feedback we received through the Road to Pluralism projects: When journalists engaged with audiences, we found that those who distrusted the media disliked overuse of adjectives and snarky headlines, and preferred clinical and sanitized descriptions of events. We also brought up a possibility that did not prove true: that readers would find objective headlines on controversial topics to be boring, and avoid them. We wanted to test these principles in an extreme case: a disproven and somewhat ridiculous (yet not uncommon) conspiracy theory about the pens used at Arizona polling places.
Using this feedback, we made three different versions of the same headline:
Snarky: “[Candidate name], Republican Primary Loser, Dabbles in Disproven Theories about Polling Place Pens”
Middle ground: “[Candidate name], Who Failed to Win Republican Primary, Claims Polling Place Pens Contribute to Fraud”
Overly balanced: “[Candidate name], Former Republican Primary Candidate, Voices Concerns about Polling Place Pens”
These factually correct headlines clearly vary in their attitude toward the candidate and willingness to condemn the false information. The first headline is snarky, using alliteration and calling the candidate a loser and their policy stance “disproven theories.” The middle ground post replaces loser with “failed to win,” and reframes the false information as a “claim” that softens the accusation: the pens merely “contribute to fraud.”
Finally, the “overly balanced” headline doesn’t imply any judgment about the conspiracy theory, merely saying a “former Republican primary candidate” has “concerns” about the polling place pens. (Maybe they’re just out of ink!) This headline is the most neutral and clinical, which is what many readers told journalists they were looking for in local news, and is also the kind of headline that those advocating for pro-democracy news coverage might condemn as “both-sidesism” that amplifies lies and reduces trust in elections.
The “overly balanced” and “middle ground” headlines were significantly more likely to be read. Seeing these headlines made our respondents more likely to choose that headline, compared to the “snarky” headline, by 7%. The “middle ground” headline was also significantly more likely to be chosen than the snarky headline, by 4.7%. These headlines, therefore, draw in substantially more readers than the headline with the strongest rebuke of the election misinformation, though we need to run more tests to determine which approach is most appealing. (Again, these results are preliminary, and we plan to run similar experiments testing the effects of different language in the lede, reader attitudes after exposure to a specific treatment, and other variations.)
Journalists face a real bind when politicians make false claims about election integrity. Simply passing on false claims to readers feels wrong, but headlines laden with judgmental language and colorful adjectives may turn off the very readers who need to read the story and see the claim debunked. If journalists lose readers to social media and partisan news because of their stridently “pro-democracy” headlines, it may ironically be the case that headlines with a softer touch are a necessary compromise for journalists looking to debunk these claims.
In other words, we found preliminary evidence supporting a much older truism: neutrality sells. A major reason for the turn toward modern journalism in the first place — larger and more politically diverse audiences that advertisers wanted to reach without offending them — may still have some merit. There will be cases where the goal of political balance pushes up against other normative concerns, such as democratic values and election integrity. We are just beginning to look into these questions experimentally, and these results will help guide our next steps. In future studies, we will examine the role of individual attributes, such as age and party identification, in greater detail.
Journalism undoubtedly plays a role in reducing polarization and strengthening democracy, but we should not throw out the old principles of balanced, marketable journalism without considering all the consequences of that decision.
More anti-polarization resources
- We created an anti-polarization checklist to be used by newsrooms. Here’s what our partner journalists said they learned while using it.
- By being more aware of and cautious about their word choices, journalists can avoid sending accidental signals about their own views on the story and about whose values the content is reflecting. Check out this look at words to avoid or be cautious with.
- Careless headlines can undermine the careful work involved in framing stories. Run your headlines through this checklist.
- Building trust and understanding starts by understanding mistrust. Here’s our guide to community interviews focused on people with low trust. And here’s what our partner journalists said they learned while using it.
Are you on a Road to Pluralism?
See what our initiative involves and find more resources on our website.
At Trusting News, we learn how people decide what news to trust and turn that knowledge into actionable strategies for journalists. We train and empower journalists to take responsibility for demonstrating credibility and actively earning trust through transparency and engagement. We’re co-hosted by the Reynolds Journalism Institute and the American Press Institute. Subscribe to our Trust Tips newsletter. Follow us on Twitter and Facebook. Read more about our work at TrustingNews.org.
