Finding Social Media Trolls with Dynamic Keyword Selection
Introduction
Many of us have come to know firsthand that social media is a hub for connection, entertainment, and information… but also a vehicle for online harassment. A 2017 Pew Survey shows that 41% of Americans have experienced online harassment, and 66% have witnessed it. This figure is only on the rise as social media platforms gain popularity.
Online trolling is difficult to address in part because of the dynamism of social media interactions. Oftentimes, abuse goes undetected due to natural topic evolution in a fast-paced discussion, as well as explicit manipulation of abusive terms by online trolls.
In 2019, we set out to understand patterns of discussion and harassment in social media data. As a first step in combating online abuse, we aimed to create a social media data collection algorithm that could capture fast topic evolution in general discussion and detect permutations of abusive keywords. In our preliminary analyses, we identified a few AI-driven tools to use for our application:
1. Closest neighbors:
- GloVe embeddings: training GloVe on a social media corpus and using cosine similarity to find the closest neighbors of a given keyword reveals new “troll” terms which crop up over time.
- k-means: running k-means clustering on embeddings for social media data reveals communities of related terms.
2. Forecasting: We found that twitter frequency data is often amenable to time series forecasting with models like ARIMA, VAR, and RNN.
Forecasts for a given keyword can help us determine whether we should continue to use the word as a query parameter for data collection. To keep our data collection results relevant, we’d like to retain trending keywords and discard keywords which are declining in popularity.
3. Meta-Data: We take into account details about a given keyword, like time since creation, sentiment, and relevance to the movement of interest. In our final algorithm, we present these details to the researcher to help them make a more informed decision about keyword selection for social media data collection.
Combining these techniques, we created a dynamic social media data streaming algorithm to track fast-changing topics and expose online trolling. I presented a poster with our ideas at the NeurIPS 2019 AI for Social Good Workshop. See our short paper for more details!
At a high level, our algorithm is as follows:
Dynamic Keyword Algorithm Overview
Begin with initial keyword set s_0 at t=0. Repeat until t=T:1. Use keyword set s_t to stream dataset K_t.2. Train GloVe embeddings on K_t to produce G_t, the embedding space of all social media dialogue. 3. For each word s in s_t, find n closest neighbors via cosine distance metric. 4. Extract relevant terms from neighbors and filter outdated terms to produce s_t+1.
The filter step in our algorithm is customizable: as of now, we factor in cosine similarity, clustering, forecasted keyword counts using ARIMA, and other metadata about the keyword (sentiment, time since creation, etc).
Algorithm Simulation
We simulated our algorithm on millions of historical tweets from the #MeToo movement, and I presented a poster with our findings at PolMeth 2020.
The timeline above shows the changes made to a constant-size keyword set while running the dynamic keyword algorithm on our historical #MeToo data. In practice, the keyword set size can vary — we keep it constant here to enable comparison to competitive baselines, as shown below.
Performance
In the figure on the left, we compare our “dynamic” monitor based on embeddings with a frequency-based “last-top” monitor, as well as a “static” monitor which retains the same keywords throughout the simulation. F1 score was calculated by comparing the 20 most popular topics in data pulled by a given monitor to the 20 most popular topics in the overall dataset. Here, we see that an embeddings-based dynamic approach can pick up topics in a fast-paced social media movement.
By utilizing cosine similarity on the embedding space, our dynamic algorithm also revealed new trolling keywords throughout the simulation, like the anti-feminist hashtags pictured below.
Next Steps
As a next step, we are fine-tuning the parameters in our algorithm and refining our time series forecasting models. For instance, we are using models like VAR to leverage correlations between related hashtags like ‘#yesallwomen’ and ‘#notallmen’. Thanks for reading, and stay tuned for more work!
