Psychological Inoculation: New Techniques for Fighting Online Extremism

Online disinformation and propaganda have evolved into one of the greatest challenges to the safety of the internet. As part of Jigsaw’s ongoing work to address extremism and misinformation, we’ve partnered with leading experts at American University’s Polarization and Extremism Research Innovation Lab (PERIL) to explore technological approaches to addressing online radicalization and proliferation of misinformation.

One of the first approaches we investigated was psychological inoculation, also called attitudinal inoculation, a technique to help people resist manipulative messages. We specifically designed this study to test psychological inoculation against common male supremacy and white supremacy messages. Psychological inoculation works by helping people build “mental antibodies” by briefly exposing them to a weakened persuasive message and thoroughly refuting it. The goal is to help individuals better recognize and resist similar misleading messages when they encounter them in the future.

Inoculation can build up people’s resistance or “mental antibodies” to encountering misinformation in the future, the way vaccines create antibodies that fight against future infection. (Source:Jigsaw)

In early 2020 we partnered with PERIL researcher Kurt Braddock, who published a foundational study of inoculation against text-based extremism, and extremism and radicalization expert Cynthia Miller-Idriss, to design a first-of-its-kind study to test inoculation against white and male supremacy propaganda focused on video and memes, which are increasingly the medium for such propaganda.

We predicted that inoculated people would experience greater psychological reactance (a combination of anger and counter arguing) when they were exposed to “scientific racism” propaganda — the use of pseudoscience to “prove” the theory that biology is a predictor of culture, ignoring evidence to the contrary. We also predicted that people would report feeling less gratification from extremist messages, would perceive the sources to be less credible, and would be less willing to support groups that promote ideas consistent with extremism.

We recruited 800 participants (majority white males aged 18–35 years olds, a demographic frequently targeted by white supremacist messaging) and showed them a short video-based inoculation message that demonstrates rhetorical techniques common to scientific racism propaganda. Participants were then split into five groups, which were exposed to videos with varying degrees of subtlety for their extremist narratives — one with blatant extremism, another more subtle video, an even subtler meme, and a control video without any extremist narratives.

A flowchart of the study design. Participants were split into five categories, where they were exposed to videos with varying degrees of subtlety for their extremist narratives. (Source: Jigsaw)

Each inoculation message contained examples of scientific racism propaganda (referred to as “microdoses”), each several seconds long. A video presenter then explained each trope, its overall message, its persuasive and manipulative function, and the negative outcomes associated with accepting it as true.

A screenshot of one of the inoculation videos produced by Jigsaw and American University’s PERIL, showing a narrator explaining that “sometimes you can identify propaganda not so much by what they say, but by how they say it.” (Source:PERIL)

Our study offered promising results: We showed that people who watched the 3-min inoculation video were less likely to support scientific racism than people who were not inoculated. This included lower levels of willingness to provide ideological, financial, logistic, and armed support to violent extremists.

Additionally, after seeing an inoculation video, participants argued against the extremist positions more frequently and their perceptions of credibility of the propaganda sources diminished.

This study offers one way to improve how people engage with propaganda online, helping them become more thoughtful about their sources, form counterarguments against racist rhetoric, and more effectively resist disinformation.

“We’re excited to be able to partner with Jigsaw on this groundbreaking work,” states PERIL’s Miller-Idriss. Future research could extend this work by testing inoculation theory across geographies, ideologies, and mediums. We’re mapping out a range of new studies to test how inoculation approaches work with different media and to counter different types of propaganda.”

According to Brian Hughes, Associate Director of PERIL, “Racism, misogyny, and extremism express themselves differently depending on the cultural context. With the help of our international colleagues, we are expanding this work beyond the English-speaking world to understand the narratives and rhetoric that drive extremism across countries and regions.”

There remain open questions about the effectiveness of inoculation techniques and this is only the first in a series of research papers we will be releasing with PERIL based on this study. Forthcoming papers will cover inoculation, extremism and online radicalization through fall 2021. New research will also cover the intersection of subversive online behavior and susceptibility to propaganda.

By Beth Goldberg, Research Program Manager at Jigsaw