We Are More Than Our Partisanship
During the most contentious election in US history, millions of social media posts reveal Americans’ shared values buried underneath intense partisan rhetoric.
After months of rancorous debates, embarrassing scandals, and relentless attacks, we’ve reached the end of an incredibly divisive presidential election. The fears of election rigging and organized voter suppression largely didn’t materialize. Voters made it clear that they want Donald Trump, and not Hillary Clinton, to be their next president.
There is much to discuss. Trump’s support from white supremacists and anti-government groups has many Americans feeling scared for their safety, while his trade and foreign policy positions have destabilized markets and spooked our allies. Political operatives and the pundit class, from the talking heads on CNN to the data-driven forecasters at FiveThirtyEight, utterly failed to predict this outcome. And in the midst of the chaos, it’s time for both sides to reconcile.
The wounds are still fresh, but beneath the vitriol, there may be hope. Our analysis found that while Americans still strongly disagree about policy issues like guns and taxes, subtle patterns in their language, especially on topics like work, family, and even government, reveal shared values between entrenched partisans. Even in the aftermath of a bitter election, we can still find common ground.
How the Analysis Works
We collected nearly 2 million tweets and Facebook comments from hundreds of thousands of Hillary Clinton and Donald Trump supporters during September and October. Using a new machine learning approach for text analysis, we compared the language in those comments and tweets to language used by the candidates in order to understand when supporters adopted their candidate’s message, when they didn’t, and how these shifts might predict or refute changes in poll numbers. We also compared supporters’ language to millions of mainstream newspaper articles to uncover where supporters were most biased in discussing the issues. Finally we compared Clinton and Trump supporters to each other, so we could measure which issues were most polarizing, and where both sides might actually agree.
How Much Do Candidates Shape How We Think?
Supporters are usually ideologically aligned with their candidate, so it’s not surprising that they adopt their candidate’s language on key campaign issues. We’re calling the degree to which supporters use their campaign’s language supporter allegiance. The Trump campaign had remarkable success in maintaining supporter allegiance — even during embarrassing news cycles, like the one focused on a leaked Access Hollywood tape in which Trump bragged about sexual assault. Though the Clinton campaign struggled to keep supporters on message, Trump supporters’ language was in lockstep with the campaign throughout September and October.
Trump’s supporter allegiance was steadfast in spite of fluctuations in the polls. However based on the aggregated polling projections modeled by popular data journalism site FiveThirtyEight, changes in Clinton’s chances of winning the election could be predicted one week in advance by measuring changes in her supporter allegiance.
In between September 22nd and September 29th, Clinton’s supporter allegiance rose 7.5%, and in the following week her projected chances of winning jumped by 23%. The week of September 29th Clinton’s supporter allegiance rose again, and the week of October 6th her odds of winning improved again. The week ending October 20th Clinton’s supporter allegiance feel by 8% , and her chances of winning quickly followed suit, dropping from 86% to 81% between October 21st and October 29th. It continued to drop until election day, when Clinton lost nearly every swing state, in spite of projections from FiveThirtyEight, the New York Times, and others that gave her a 70–90% chance of winning the presidency right up until the polls closed.
Supporter allegiance shifts for both candidates depending on the issue. For example Trump supporters showed strong allegiance on topics like “money” — usually in reference to Clinton’s relationship with Wall Street — and “immigration.” Clinton supporters showed the most allegiance when discussing “women” and “taxes.”
Turns Out We’re All Biased
The key to understanding voter attitudes and partisanship is bias, and there’s plenty of bias in how each side discusses the key issues facing our country. Our analysis determines bias based on the context in which Trump and Clinton supporters used relevant keywords, and compares that context to mainstream language.
For example, in typical English, like a New York Times article, the word “abortion” is used in a statistically similar context to words and phrases like “pro-life,” “legalized,” and “gay marriage,” which indicates that journalists typically write about abortion in the same way as they write about any social issue. By contrast, amongst Trump supporters “abortion” is statistically similar to “TPP,” “Obama,” and “ISIS” (i.e., they write about abortion in a similar way to other political topics about which they have a strong, negative position), while Clinton supporters discuss “abortion” in the same context as “reproductive” and “healthcare” (i.e., “reproductive rights” is synonymous with “abortion rights”).
Neither side is neutral, and the distance between how each side discusses the issue quantifies a deep partisan divide.
This is true for many of the topics that were popular with each campaign and their supporters during the election. For example Clinton supporters used particularly biased language when talking about guns and trade, while Trump supporters showed more bias when discussing taxes, women, and work.
Bias isn’t necessarily bad. It might indicate an especially passionate position on a topic, or a point of view that’s out of sync with the mainstream — for better or worse. In either case, a strong bias doesn’t mean that a group is wrong about an issue, but it does indicate that they don’t discuss that issue using objective language.
Shared Bias Reveals Shared Values
Even after an exceptional election, the only way forward is together. Fortunately, at least on some topics, supporters of both Hillary Clinton and Donald Trump seem to see eye to eye. We know this because supporters from both sides discuss these topics in very similar ways, which — just like the discussion of “abortion” or “taxes” indicates partisan divide — in this case indicates shared values.
It appears that we’re most likely to agree when we talk about ourselves and our communities — for example words like “people,” along with topics like “family” — and when we talk about work. One possible explanation is that, even when we disagree about policy issues like trade, our everyday cares and concerns are often the same.
This is the beginning of an uncertain time for the United States. We may disagree, we may fight, and we may struggle to see past our partisan differences over the next four years. But even when we disagree, let’s remember that we’re all human beings — we all work, we all love our families, and we all want a government that serves the American people. As the sting of this difficult election starts to fade, we have no choice but to build on these common values, and a find a path forward together.
Notes: Given my community, I assume the majority of people reading this will lean left. For the record, I supported Clinton and was strongly opposed to Trump’s candidacy. However in spite of the many valid arguments made against that candidacy during the campaign, he still won a fair election. I don’t like the campaign he ran, and will likely struggle with many of his policy initiatives, but he’ll still be our president starting in January. I am sincere in my hope that he is a better president than I am expecting. If you’re a Trump supporter, I sincerely want to understand your vision for the country.
We’ll publish more details on how we collected data for this analysis, and the technique we’ve developed for identifying bias. In the meantime, feel free to reach out to me via email at jonathon@newknowledge.io or on Twitter @jonathonmorgan.
