Revisiting The Famous 2016 “Economic Views vs. Social Views” Scatterplot

Introduction

After the 2016 election, Lee Drutman and the Voter Study Group did an analysis of public opinion and its connection to the election. Their most-discussed conclusion was that Trump won by winning “populists”, or voters who were economically left-wing but socially right-wing. This analysis became very popular. The main chart, a scatterplot of respondents by “economic issues” and “social issues”, became even more popular:

Years after the original article, I still see this chart regularly on Twitter. I recently saw it in this Tweet, which claimed it showed “the ‘economic right, socially left’ suburban voter is not prevalent outside of the DC/NY/CA Republican base, mostly on Twitter”:

I don’t begrudge the chart its popularity and influence. There is a lot that I like about it.

For example, I have a general principle that graphics should show the most granular data you possible, so I really like that the chart has a dot for every individual respondent.

Moreover, the general message that “Trump won in 2016 by winning economically liberal and socially conservative voters” is not, I think, a bad summary of the specific dynamics of that election. This was a message conveyed by both the above chart and other charts in the article such as: this one (which uses “Populist”, “Conservative”, “Liberal”, and “Libertarian” for the four quadrants of the original scatterplot):

However, I have a couple of issues with the chart, and even more, with the discourse that followed.

First of all, I think the graphic itself could have conveyed a bit more information about its content and assumptions. That’s going to be my focus for the rest of this article.

Second of all — this may require follow-ups — the graphic seems to conflate two distinct questions.

One question is about the structure of American public opinion itself. “How do social issues and economic issues fit together”, “which issue positions are popular”, “what coherent subgroups exist”, etc.

The second question is about the 2016 election — “what predicted Obama/Trump voters”, “what were the characteristics of swing voters”, etc.

These questions might be related but I do think they are distinct and probably require separate graphics and analysis.

What’s Missing From The Graphic

What information do I think the should the chart have added?

I’ve always wanted the graphic itself, and not just the accompanying article, to be more transparent about how exactly “economic dimension” and “social/identity dimension” were defined, or about what the scores actually meant. That is, I want my version of the graphic to convey exactly what issue positions make someone a “populist” or a “libertarian” in the sense of the original chart.

This is not just me being pedantic. If you write an analysis based on your own idiosyncratic definitions of common phrases, and then make a graphic only labeled with those common phrases, then that graphic will probably be shared independently of your analysis, and people are probably going to bring their own preconceptions to it.

I believe the Tweet I quoted above was someone bringing their own preconceptions to this graphic. The chart itself didn’t say anything about “suburban voters” or “the DC/NY/CA Republican base”, but those are the preconceptions that person had when they saw “economic dimension” and “social/identity dimension”.

For another example, the chart was so popular that a similar (and similarly opaque) chart was attempted after 2020:

A popular Tweet about that chart claimed “like three-quarters of the Dem caucus are in the [“Libertarian”] bottom right with 5 percent of voters”:

Again, I think this is unlikely to be true, at least literally true. Looking at the the actual questions that went into it, the “Progressive (economic) positions” are mostly standard Democratic positions like raising the minimum wage, probably held by a majority of the Democratic caucus:

I can’t read Cooper’s mind. But I imagine he had his own pre-existing definition of “social progressive and economic conservative”, and perhaps his pre-existing definition does indeed fit “like three-quarters of the Dem caucus”. So, when Cooper saw a chart labeled “more progressive on cultural/social issues, more conservative on economic issues”, he probably assumed it matched that definition of his.

And why not? It’s not really his fault; the chart he saw didn’t say otherwise.

Replicating The Scatterplot

So, can we try to replicate the chart, in a way that includes more context?

The data for the chart is from the Democracy Fund Voter Study Group survey and is helpfully freely available.

Drutman wrote that he used several questions to create “indices” around various issues, and then combined those into the “Economic Dimension” and “Social Dimension”:

The 12 dimensions provide nuance, but for simplicity’s sake, it is easier to combine these indexes into the two main dimensions that organize public opinion: questions of economics and questions of culture/social/national identity.

To do this, I created two new indexes:

An economic liberalism-conservatism index (which combines views on the social safety net, trade, inequality, and active government)

A social/identity liberalism-conservatism politics index (which combines the moral issues index plus views toward African-Americans, immigrants, and Muslims).

I identified 12 questions that apparently went into the “Economic Dimension” and 12 questions that apparently went into the “Social Dimension”. Following the original analysis, I gave each respondent a “score” for each question from -1 to 1.

To illustrate what this means in practice, “imiss_m_2016”, or “Issue importance — Social Security” had the “most liberal” answers on average, with 5682 answering “very important” (scored as -1), 1844 answering “somewhat important” (scored -1/3), and only 280 (scored +1/3) or 65 (scored +1) answering “not very important” or “unimportant”:

Conversely, “race_overcome_2016”, or basically “Blacks should overcome without help”, had the “most conservative” answers on average, with “Strongly Agree” scored +1 and “Strongly Disagree” scored -1.

I then averaged those scores to get the subtopic “indices”, then took the averages of those indices to get the “economic index” and “social/identity index”.

The result, which you can see here, matched the original scatterplot very closely. It did not match exactly. But I think it matched closely enough that the differences are probably minor things about which blank values to exclude and so on:

Anyway, exactly replicating the chart is not quite as important as illustrating the information I wanted to add.

Adding Information

What information did I want to add to this chart?

I wanted the specific questions included on the chart. I therefore added input boxes for the 12 “Economic Dimension” questions and the 12 “Social Dimension” questions. This also allows for different choices of weights than the ones implicit in the original chart:

I also included the Presidential numbers by quadrant for 2012 and 2016, both to check against the original analysis and out of their own interest.

(Along with an option to highlight the Obama/Trump voters:)

You may recall my principle that a graphic should include data at the most granular possible level. I therefore also added a dropdown table so you can see the individual answers for each individual respondent and how they contribute to the scored dimensions:

To give a little more additional context, I also added “average scores” for each question, as described above.

Re-Interpreting The Scatterplot

Does the added information I wanted make the scatterplot easier to interpret?

For one thing, I hope my version clarifies that most of the questions that went into the “social/identity” dimension were actually about Black Americans, Muslims, and immigration. Only three of the twelve questions in the “Social Dimension” are really about “social issues” as I would usually use the phrase (issues like gay marriage, abortion, and transgender acceptance). Indeed the article itself defines a “populist” as “liberal on economic issues, conservative on race issues” (my emphasis).

How “Important” Are “Importance Of Medicare And Social Security”?

I had also wondered how much all of these results were driven by the questions about whether Medicare and Social Security were “important to you”. These were by far the most heavily-”progressive” questions out of the 24, so they will naturally push respondents into the “economically progressive” quadrants.

What happens if you zero out those questions? Suddenly Trump voters become a bit less “split by economic issues”:

Using the tooltip menus, we can easily find some of these respondents that end up in the “populist” quadrant if and only if you include those two questions, but otherwise end up in the “conservative” quadrant:

This particular respondent, a Romney/Trump voter, thinks the wealth distribution is fair, they don’t want to raise taxes on the rich, they think the government regulates business too much and that the free market is better…but they think Social Security and Medicare are “very important” or “somewhat important. Along with their answers on trade, this would be enough to put them in the “populist” quadrant if we give those questions their original weights. Without them, the respondent is in the “conservative” quadrant.

Is this respondent they “really” a “populist”, or “really” a “conservative”? Well, that’s up to you and your definition of “populist” (or of “conservative”). Or maybe it would be revealed by the respondent’s answers to other questions in the survey beyond these 24. But either way, I’m glad we have the information available to ask.

“Social Dimension” Or Racial Dimension?

You may also be wondering about the respondents in the so-called “libertarian” quadrant. We can explore them further as well:

In this case, you can see that this particular respondent, a Clinton/Obama voter, does not really fit at least my own definition of “economic conservative and social liberal”. They’re against gay marriage, think trans people should use the bathroom they were “born into”, and have a middle-of-the-road opinion on abortion.

Most of what pushes them into the “libertarian” quadrant is that they’re pretty favorable to Muslims, they “somewhat oppose” a ban on Muslim immigration, they think that illegal immigrants mostly make contributions rather than being a drain on society, etc. Remember, in sheer volume of questions, it’s really a racial issues dimension, more than a social issues dimension.

Indeed, if I overweight the “social issues”, then a few Obama/Trump voters actually shift to the “progressive” quadrant”, like this respondent:

While if I instead zero out the “pure social issues”, then Obama/Trump voters become a tiny bit more heavily “populist” than they even were initially:

That doesn’t mean that “social issues” aren’t an interesting dimension in their own right. And it doesn’t mean there weren’t a significant number of “socially conservative” Obama/Trump voters.

(Although I do think the race/immigration questions were a little more predictive of that flip. Not to overstate the point; it’s all pretty correlated under the hood, and the gay marriage and transgender questions are predictive in their own right. But, out of 254 Obama/Trump respondents, I have 188 Obama/Trump respondents in the “just immigration/race populist” quadrant, as opposed to 172 in the “original populist” quadrant, and just 125 in the “just pure social issues populist quadrant”. Which is probably very dependent on ties and floating point arithmetic stuff, especially the last one…)

Anyway, perhaps the effects of these changes can be a reminder that the dimensions in the original chart are the constructed product of specific choices, and those choices can influence the results.

Conclusion

Again, I don’t want this to be read as a “takedown" or refutation of the original analysis, which seems to be pretty sound.

Mostly, I want to remind everyone to dig into the data themselves, or at least read the methods and definitions, before drawing conclusions based on a chart they saw somewhere.

Notes

Racial issues also dominate the “cultural/social” axis of the 2020 version of the scatterplot, with three of the five questions:

There is an odd nuance with one of the trade questions, which asks if trade deals increase or decrease “prices”. You’d think “increase” would be the “anti-trade” answer and “decrease” the “pro-trade” answer, but in fact most people who answered both answered “increase/decrease prices” and “increase/decrease jobs” the same way. I used the “logical” answer and not the “correlated” answer for the score to that item, but I can see the argument either way…and have actually gone back and forth.