Empowering Digital Counter-Messaging Teams to Challenge Radical Narratives Through Machine Learning Databases
How people consume and share digital messaging matters. The actions taken by Cesar Sayoc in October 2018 make it clear why. Sayoc was arrested on October 26, 2018 after sending pipe bombs to many prominent critics of President Donald Trump. No one was hurt as a result of Sayoc’s actions. However, Sayoc’s terrorist behavior sparked interest in his online presence, and his words on social media painted a radicalized picture. In a New York Times article, journalists archived Sayoc’s social media posts before Sayoc’s accounts were taken down by Facebook and Twitter. Kevin Roose writes:
For years, these platforms captured Mr. Sayoc’s attention with a steady flow of outrage and hyperpartisan clickbait and gave him a public venue to declare his allegiance to Mr. Trump and his antipathy for the president’s enemies.
In the article, Roose describes dozens of posts in which Sayoc shared radicalizing messages about Islamist terrorism, illegal immigration, and anti-Clinton and anti-Obama conspiracy theories. Notably, Sayoc continued to post as the FBI manhunt took place for the serial bomber. Roose describes Sayoc’s tweet posted Wednesday, October 24, two days after the first bomb arrived at the home of billionaire investor George Soros and two days before Sayoc’s eventual arrest:
[Sayoc] sent a tweet to the Twitter account at the website TMZ that criticized Andrew Gillum, the Democratic candidate for governor in Florida, for raising campaign funds from Mr. Soros and other liberal donors.
“$500,000 Soros puppet,” read the text over the image.
Although it is impossible to determine the exact effect of radicalizing content on Sayoc, it is clear that social media played a role in his worldview and ultimate decision to seek to hurt who he saw as his enemies. It is with examples like this in mind that made my experience in Hacking for Defense (commonly referred to as “H4D”), a class taught at the University of Southern California (USC), so meaningful.
Hacking for Defense is a semester-long course which pairs governmental defense organizations working on tough problems with students willing to try and tackle those problems. Along the way, students are mentored on the lean start-up methodology, which gives students a framework in which to think about their problem and launch a solution. Our team was composed of Brian (the tech guy majoring in computer science), Ben (the “jack of all trades” studying design, engineering, and business at the Iovine and Young Academy), Mimi (the sophomore, but the most impressive of the bunch) and me (the liberal arts major).
At the start of the semester, our team found ourselves in a class full of graduate students, partnered with the State Department’s Bureau of Countering Violent Extremism (CVE) on a “boil the ocean” sort of problem: how might we identify, reach, and intervene in the lives of youth at risk of radicalization? With the help of our problem sponsor, class instructors, and 70 interviewed stakeholders involved in the CVE space, we ultimately narrowed our scope to a product that would empower digital counter-messaging teams across the globe to challenge radical narratives online.
We began our journey by researching online radicalization. We learned that terrorist organizations and white nationalist groups are some of the first groups to have secured territory online, devoted to recruiting individuals to their cause. These groups spread propaganda to youth via online platforms like Twitter and Facebook, while also delivering personalized recruitment messages through encrypted messaging platforms like Telegram.
This is where CVE, and the work of the State Department Bureau we partnered with, comes in. CVE is an amalgamation of governmental and civilian initiatives that seek to counter and prevent violent extremism by addressing the range of possible causes. Addressing the heart of root causes, we find family prevention and community initiatives like peace-building and religious tolerance workshops. Next, social workers, psychologists, law enforcement, religious leaders, and academics form interdisciplinary teams, creating individualized intervention programs. When extremist thought becomes extremist action, there are more reactionary responses, such as security intervention. Our CVE solution would need to focus somewhere in between all these efforts, but we were unsure where exactly.
The first academics and researchers we spoke to, like Dr. Erroll Southers at USC, suggested the United States and Germany as potential beneficiaries of our solution, given the many white nationalist groups that have sprung up recently. We were passionate about this issue, however, after discussions with the State Department, we realized we needed to focus on international developing countries. So, by tracking key metrics that factored into a country’s need for an intervention program like internet penetration, the impact of terrorism, and CVE funding, we narrowed our focus to the Philippines and Malaysia, two countries we believed were ripe for new CVE developments.
The first week after choosing these countries, we had no idea how to get in contact with folks on the ground, already working on the issue. Ben and Mimi spent many nights cold-calling social workers, governmental organizations devoted to CVE, and religious leaders abroad, but no one wanted to talk to us.
When we finally did get in touch with the Southeast Asia Regional Centre for Counter-Terrorism (SEARCCT) team working on this very issue in Malaysia, we learned the CVE space in these countries is not dominated by intervention teams, but rather by counter-narrative teams. In developing countries, there are not enough resources to hire dozens of government officials, social workers, and community leaders to create end-to-end intervention infrastructure programs. Instead, an online, scalable solution is more feasible because this is where smaller teams can arguably have a greater impact.
Digital counter-messaging teams craft alternate, positive narratives that reach youth who encounter terrorist narratives online. The process of crafting these messages is lengthy and requires lots of collaborative work. First, the social media analyst tracks terrorist messages online, collects screenshots of those messages, and organizes them according to helpful tags like language, narrative type, or grievance expressed. From there, the rest of the team can retrieve messages from the analyst via email to conduct their various analyses.
One of those team members is the narrative writer, who, depending on the radicalizing message, crafts an appropriate counternarrative. That counter-narrative is then taken to the content producer who turns the message into digestible online content like a short video or popular meme. The content is pushed out onto social media platforms, and a social media marketer tracks the success of the narrative.
We realized there was a huge hurdle at the first step of this process, where the social media analyst is forced to spend entire days screenshotting Twitter, Telegram, or Facebook posts and sorting them into literal desktop folders according to certain categories. If another team member wants to study a certain message, the social media analyst must email the screenshot to that team member. It’s a huge time sink for that one person, but also the entire team: the narrative writer, content creator, and marketer must rely on the analyst’s organization and emails to search through messages.
Our solution, which we called “Turnstone,” would automate the manual scraping and categorization in two-steps, in turn giving time back to the analyst to further analyze these messages. (1) Instead of screenshotting messages, Turnstone would pull from Twitter and Telegram’s APIs to web scrape and retrieve extremist content. Then, the analyst would tag and categorize the messages using Turnstone’s interface, ultimately resulting in a tagged and recorded data set of extremist messages.
(2) Using natural language processing methods and the tagged database, we would create a machine learning algorithm that would sort and tag all the messages, fully automating the sorting of content. We learned that this sort of machine learning is already being done academic researchers like Adam Badawy and Emilio Ferrara, the authors of Predicting Online Extremism, Content Adopters, and Interaction Reciprocity and The Rise of Jihadist Propaganda on Social Networks. After conversation with the pair, we believe it is theoretically possible, with a large enough data set, to teach natural language processing machines how to categorize and tag social media posts according to many of the tags counter-messaging analysts use today.
The end result is a searchable database with an interface like the one below, where a counter-messaging team of users could sort messages by location, time, platform, language, level of radicalization, narrative type, and type of grievance expressed — the same helpful categories that analysts’ use now. With a majority of their time freed up, analysts can now refocus on the more complex analysis of terrorist narratives, and the entire team no longer relies on the analyst to easily search and make sense of extremist messaging online.
Over the rest of the semester, we began conversations with two other digital terrorism research organizations in Singapore (International Centre for Political Violence and Terrorism Research) and the United Arab Emirates (Sawab Center) and validated that their counter-messaging teams followed similar processes in crafting these online narratives. This meant that our software product could potentially be applied in teams across the globe, making Turnstone valuable to a larger market of customers.
… And that’s where our 15-week sprint in H4D left us. Ultimately, we presented a proof of concept to our State Department sponsor and a host of defense industry professionals including Rt. Hon. General David Petraeus. We received lots of great feedback and opportunities to collaborate with other teams. There is still a lot to be done, but we hope our customer discovery and minimum viable product highlighted to the State Department the importance of technological tools that can make counter-messaging teams more productive. Although my three teammates and I are headed down paths different from this particular project, I can say that the entire team is excited about potential opportunities in civic and government tech down the road.
