How do we quantify polarization?
Alex Zhao
Introduction
In the United States, controversy across the political spectrum often centers on a perceived increase in polarization and a lack of substantive policy analysis by candidates. Interestingly, research suggests that people who voted for Donald Trump in 2016 are unlikely to change their minds regardless of facts and filter all new information through a partisan lens, confirming suspicions of rampant polarization (Aschwanden, 2016). It is thus crucial to have an objective and consistent metric to analyze distinctions between candidates and their policy positions, especially as the 2020 election looms.
In the US, polarization is the separation between the Democratic Party and the Republican Party on policy positions. In order to test levels of polarization in the current elections, we can compare differences between the two parties on specific policy issues. I chose two categories — foreign policy and domestic policy. The foreign/domestic distinction is relatively clear-cut and may reveal insights about the modern party system. We thus end up with two hypotheses:
Hypothesis 1: Democratic candidates and Republican candidates significantly differ in their integrative complexity levels on foreign policy.
Hypothesis 2: Democratic candidates and Republican candidates significantly differ in their integrative complexity levels on domestic policy.
Integrative complexity (IC) is a measurement of information processing and the “elaboration and complexity of any given information or thought” (Thoemmes and Conway, 2007). These thoughts must be captured in a way that can be practically evaluated — for instance, spoken word or written language of any kind. IC can be defined by two primary elements: differentiation and integration. Differentiation, which is a prerequisite to integration, is the perception of multiple ideas, alternatives, or approaches to thinking about an issue. Integration is the perception of relationships between these ideas. Importantly, integrative complexity is not concerned with the content of the speaker’s argument, but its structure; complexity is thus not directly tied to political orientation in any way.
Extensive research has been done on the relationship between complexity and effective leadership, including studies on the Cuban Missile Crisis, surprise attacks, and peace negotiations (Guttieri et al., 1995; Suedfeld and Bluck, 1988; Tibon, 2000); however, research has been lacking regarding the distinctions in IC between American politicians. The purpose of this study is to analyze presidential candidates’ integrative complexity based on a set of interview transcripts. In contrast to previous research, which has focused on prepared speeches and addresses, this study analyzes 77 interviews from presidential candidates for complexity. I test for distinctions between Republican and Democrat presidential candidates and differences in complexity between parties on foreign policy and domestic policy.
Literature Review
As a metric of information processing, integrative complexity (IC) has been applied to a broad range of cases involving political decision-making and leadership (Conway, Suedfeld, & Clements, 2003). Conway, Suedfeld, & Tetlock (2001) found that a drop in IC was associated with an increase in the likelihood of violent conflict, while high IC among political leaders predicted peaceful resolution. Leaders with lower levels of complex thinking were more likely to resort to military action, rather than compromise. Several other studies further suggest that IC levels are extremely relevant for explaining the policy outcomes of leaders (Suedfeld, Corteen, & McCormick, 1986; Tetlock, Peterson, & Berry, 1993). By analyzing transcripts of interactions among key decision-makers during the Cuban Missile Crisis, Guttieri et al. (1995) found that American officials maintained moderate levels of IC; mean IC scores increased when leaders were in private, implying that the intense scrutiny of the public eye suppressed complexity.
As a group, presidential candidates have been thoroughly studied by psychologists and political scientists alike. Thoemmes and Conway (2007) identify both intrinsic and extrinsic reasons for studying presidents: not only are they particularly interesting due to their monumental achievement, they also function as symbols for the pinnacle of political success and thus can also serve as subjects for the study of the party system at large. Learning about the candidates may allow us to extrapolate information about the party’s message, the changing goals of party members, and the structure of the parties themselves.
Beinart (2008) traces the development of party divisions in the context of foreign policy from 1948 onward. In 1947, Senator Vandenberg asserted that “politics end at the water’s edge”; in other words, that the political turmoil and divisiveness of American government should not apply to foreign policy because the parties must represent a united front to other countries. For much of the Cold War, there existed a broad bipartisan consensus on the overall strategy for U.S. foreign policy. However, as time went on, divisions between the two major parties became more pronounced and obvious. From Vietnam to the War on Terror, Democrats and Republicans gradually drifted apart. The War on Terror began as a nearly unanimous effort, with the PATRIOT Act and campaign in Afghanistan supported by all but one Democrat. In contrast, the U.S. invasion of Iraq has become highly divisive issue that characterizes the difference between Democrats and Republicans on foreign policy. Democrats are now framed as “doves” that are generally soft in foreign affairs, while Republicans are the “hawks.” Of course, there are exceptions (the libertarian wing in particular comes to mind). In general, though, Republicans have become more hawkish and likely to support military force; the Pew Research Center (2005) found that the partisan gap on military versus diplomatic methods to keep the peace grew from 16 percent in 1999 to 44 percent in 2004.
Materials and Methods
Previous studies of presidents and politicians have included State of the Union addresses and other prepared speeches (Thoemes and Conway, 2007; Tetlock, 1981). In this paper, we have chosen to analyze interview transcripts for a few reasons. First, prepared speeches may not be crafted by the candidate. Politicians often heavily rely on speechwriters for assistance, thus making IC an analysis of the candidate’s staff rather than the politician herself (Einhorn, 1982). Second, speeches are usually prepared relatively far in advance, while interviews force a greater degree of spontaneous thinking. Thus, interviews may be a better representation of candidates’ complexity than speeches.
I used interviews from Republican and Democratic candidates from the 2016 election cycle that stretch back to July 2015, with the exception of Sarah Palin’s interviews from 2008. Palin’s interviews were included to improve representation of Republican women in the data. The transcripts were stripped of all metadata and interviewer participation, containing only the candidate’s words. Only candidates included in official party debates were studied, since other politicians did not engage in extensive interviewing. Four groups of trained coders graded up to 35 transcripts. Each interview was split up into segments of 150 words each. I categorized each segment as dealing with foreign policy, domestic policy, or neither. Segments that discussed neither foreign nor domestic policy were excluded from my analysis of topic-specific complexity, but included in the evaluation of candidates’ IC overall.
As per convention in qualitative analysis, each group randomly selected five segments from each interview for coding. Individual coders within groups analyzed interviews with no knowledge of other coders’ scores to minimize intercoder influence, with the exception of a few interviews.
Each group calculated its average IC score for each candidate. I then weighted each group’s average based on the ratio of its Krippendorff’s alpha, a measure of reliability, to the average alpha value. The average value of Krippendorff’s alpha was .65. Groups with a value above this average received a higher weight, while groups with a value below .65 received less weight when calculating the final IC scores.
To determine whether the difference between Republican and Democrat candidates on policy positions is statistically significant, I used an unpaired t-test. I treated party affiliation (Democrat, Republican) as the independent variable and IC score as the dependent variable. Group 1 is the Democratic Party and Group 2 the Republican.
Results and Discussion
Weighted averages of each candidate can be found below. Democratic candidates had significantly higher scores than Republican candidates. When comparing aggregate Democratic and Republican scores, Democrats were, on average, 0.55 higher on the scale. Democrats average 2.19 and Republicans 1.64.
Figure 3 displays the mean IC scores, segmented into party affiliation and policy topic (foreign versus domestic policy). Even a cursory glance at the results seems to confirm both hypotheses. Democrats have higher mean scores on both foreign and domestic policy. However, direct comparison of aggregate IC values does not demonstrate a true relationship.
After segmenting for the foreign/domestic policy divide, t-test results (see Figure 4) show a weak correlation in the arena of domestic policy (p = 0.054). Although not statistically significant, this may show that Democrats are only marginally more complex on issues of domestic policy.
On the other hand, in foreign policy, the difference between the two parties is both larger and extremely statistically significant. With a p-value of less than 0.0001, the t-test shows that Democrats are, on average, 1.01 higher on the complexity scale.
So, why is it that Republicans have lower complexity on foreign policy, but Democrats do not? A possible reason is that challenging parties focus much of their attention on criticizing the incumbent president and his policies, leaving little room for the discussion of substantive policy. Indeed, political journalists in 2016 noted that the current Republican Party seemed entirely disorganized, with the exception of its unanimous criticism of President Obama’s administration (Riehl and Frisk, 2015). This fact may show up during interviews when Republican candidates spend much of their time attacking current policies and offering only cursory analysis of their own policies. The notion that challenging parties would display lower levels of complexity seems highly intuitive; why, after all, would candidates discuss their own policies (which may open them up to criticism) when they could simply denounce the current administration to gain support?
The Republican campaign manifesto on foreign policy appeared to have subscribed to the line of reasoning described above. Throughout the Republican debates, a theme unifying the candidates on foreign policy was “Obama is weak, I am strong” (Shapiro, 2015). The adage is simple and clear-cut: Republicans represent force, while Democrats represent weakness. This basic vein of thought underlies a majority of Republican positions on foreign policy. The reliance on criticism of the incumbent president may be a useful campaign strategy, but could result in lower complexity. Nevertheless, the challenger/incumbent distinction does not explain why Republicans and Democrats are relatively similar in terms of complexity on domestic policy. This is an interesting theory to explore, especially with a Republican president as the incumbent candidate.
Another explanation could be that the strain of populist nationalism that has dominated the Republican primary tends to favor simplistic and reductionist solutions to international problems and more complex solutions to domestic issues. In particular, Donald Trump’s campaign has emphasized an isolationist approach to foreign policy that treats American interests as paramount. Trump has repeatedly spoken in favor of economic protectionism, criticizing free trade deals such as the Trans-Pacific Partnership as harmful for American workers. Trump’s populism is not unique to the United States; several European countries have also seen a rise in populist parties that advocate protectionist and isolationist approaches to international problems, a phenomenon exemplified by Britain’s vote in favor of leaving the European Union (The Economist, 2016). Trump’s variety of populism may lend itself to less complex understandings of foreign affairs, since the generic reaction to an international conflict would be that America should not get involved in the first place. In short, most global problems are “not our problem.”
Populism is also clearly tied to the challenger/incumbent distinction. Bonikowski and Gidron (2016), in a study of presidential elections from 1952 to 1996, found that populism is associated with challenger candidates due to their political outsider status. Incumbents, as symbols of the establishment and elite politics, are unable to employ the anti-establishment rhetoric of populism.
Limitations
Any qualitative analysis of individuals’ thought processes through written word is bound to have limitations, a few of which are discussed below.
First, politicians may adapt their complexity level according to the audience and news network where the interview is conducted. Since varying audiences and interviewers have differing tastes and styles, candidates may alter their thinking and, subsequently, their complexity from interview to interview. For instance, a candidate may change their rhetoric or framing of an issue when on CNN since they are familiar with the interviewer’s personal position on that issue. The candidate may try to avoid a particular line of reasoning by using evasive rhetoric and ambiguous language, all of which may influence complexity scores.
Second, since the division of transcripts into analyzable segments is an automated process, segments are often incomplete thoughts. Sentences are cut off and ideas fragmented; since fragments of a policy idea cannot be scored, automated division of transcripts loses a non-negligible amount of data. That further decreases a sample size which is already very small.
Third, segmentation based on policy topic does not account well for diverse topics discussed during interviews. Other topics may concern recent Twitter feuds, personal history, or other unrelated political gossip. This problem suggests that IC may not be the optimal tool to analyze interview transcripts. These issues must be balanced with the effects of ghostwriters and preparations on speeches.
Lastly, the generalizability of this study is limited due to the very specific nature of presidential candidates’ interviews. They are only given in a particular window of time every four years or so. The candidates that are granted a national-scale interview have already achieved prior success in fundraising, name recognition, and polling. The results of the present study cannot be extended to apply to the general population or even all politicians.
Conclusion
As another presidential election looms, we prepare once again for the intense political clashes between candidates and the accompanying discussion about whether the divide between blue and red America has worsened. Here, I have tried to provide a quantitative understanding of the divide between elites.
This study has analyzed differences between presidential candidates in the United States by studying the complexity of interview transcripts. Results show strong evidence that complexity patterns differ from party to party. Democrats are marginally more complex on issues of domestic policy than Republicans, but substantially more complex on foreign policy. Multiple factors may explain this phenomenon — the incumbent/challenger distinction, the particular message of the Republican Party in this election cycle, or growing polarization that has led Republican foreign policy to become increasingly hawkish and hence less complex.
Of course, IC scores for these presidential candidates do not necessarily predict their IC once in office. In fact, studies of politicians found that presidents have lower integrative complexity during the election cycle (perhaps to clarify their message and distinguish themselves from other candidates) than during their tenure in office (Tetlock, 1981). In other words, IC levels substantially increase after candidates enter office.
Nevertheless, these findings reveal useful insight about the ways that politicians think about policy as well as the limitations of integrative complexity as a metric for analyzing interviews. Future research should study the causes of difference in complexity between the two parties, especially in the realm of foreign policy.
Acknowledgements
I performed this research at the Ethics Center at UC Irvine. I’m grateful to Professor Kristen Monroe and Jen Jones for their help during the process.
A version of this paper was presented at the International Society of Political Psychology conference in 2017.
References
Aschwanden, C. (2016, July 20). There’s probably nothing that will change Clinton or Trump supporters’ minds.
Beinart, P. (2008). When politics no longer stops at the water’s edge: Partisan polarization and foreign policy. In Wilson J. & Rauch J. (Authors) & Nivola P. & Brady D. (Eds.), Red and blue nation?: Consequences and correction of America’s polarized politics (pp. 151–184). Brookings Institution Press. Retrieved from http://www.jstor.org/stable/10.7864/j.ctt6wpdqn.7
Bonikowski, B., & Gidron, N. (2016). The populist style in American politics: Presidential campaign discourse, 1952–1996. Social Forces, 94(4), 1593–1621.
Conway, L. G., III, Suedfeld, P., & Tetlock, P. (2001). Integrative complexity ad political decisions that lead to war or peace. In D. J. Christie, R. V. Wagner, & D. A. Winter (Eds.), Peace, conflict, and violence: Peace psychology for the 21st century.
Conway, L.G. III, Suedfeld, P., & Clements, S.M. (2003). Beyond the American reaction: Integrative complexity of Middle Eastern leaders during the 9/11 crisis. Psicología Política, 27, 93–103.
Einhorn, L. J. (1982). The ghosts unmasked: A review of literature on speechwriting. Communication Quarterly, 30(1), 41–47.
Guttieri, K., Wallace, M. D., & Suedfeld, P. (1995). The integrative complexity of American decision makers in the Cuban Missile Crisis. Journal of Conflict Resolution, 39, 595–621.
Pew Research Center. (2005, January). Politics and values in a 51%-48% nation.
Riehl, J., & Frisk, D. B. (2015, June 21). GOP presidential candidates: The more the scarier. Real Clear Politics.
Shapiro, J. (2015, August 7). The Republican foreign policy manifesto: Obama is weak and I am strong.
Silver, N. (2016, May 18). How I acted like a pundit and screwed up on Donald Trump.
Suedfeld, P., & Bluck, S. (1988). Changes in integrative complexity prior to surprise attacks. Journal of Conflict Resolution, 32, 626–635.
Suedfeld, P., Corteen, R. S., & McCormick, C. (1986). The role of integrative complexity in military leadership: Robert E. Lee and his opponents. In G. Y. Nogami (Ed.), Special issue on military psychology, Journal of Applied Social Psychology, 16, 498–507.
Suedfeld, P., Tetlock, P. E., & Ramirez, C. (1977). War, peace, and integrative complexity. Journal of Conflict Resolution, 21, 427–442.
Tetlock, P. E. (1981). Pre- to postelection shifts in presidential rhetoric: Impression management or cognitive adjustment? Journal of Personality and Social Psychology, 41(2), 207–212.
Tetlock, P. E., Peterson, R. S., & Berry, J. M. (1993). Flattering and unflattering personality portraits of integratively simple and complex managers. Journal of Personality and Social Psychology, 64(3), 500–511.
The Economist. (2016, July 30). Drawbridges up. The Economist.
Thoemmes, F. J., & Conway, L. G., III. (2007). Integrative complexity of 41 U.S. presidents. Political Psychology, 28(2), 193–226.
Tibon, S. (2000). Personality traits and peace negotiations: Integrative complexity and attitudes toward the Middle East peace process. Group Decision and Negotiation, 9(1), 1–15.
