We Built This City on Bots & Trolls
Looking into Twitter trends found from data on Bot Sentinel
Both AI-driven bots and human trolls have been found spreading false information about really important topics, especially in the area of politics. Since 2016, social media’s influence on politics has been heavily discussed and researched.
Bot Sentinel is a site that launched in 2018 that is working to help fight against these kinds of accounts. They have trained a machine learning model, using thousands of accounts and millions of tweets, that can classify problematic accounts with 95% accuracy. On their site, you can find details on hundreds of thousands of accounts. One of the details I found interesting was the date the accounts were created; I wanted to see if there was any increase in account creation during the 2016 election and now. What I found were very distinct patterns, and a huge spike in those accounts over the past couple years.
How Bots are found and scored
Each account on Bot Sentinel has a score that indicates the level of threat it is. According to their site,
“We rate accounts based on a score from 0% to 100%, the higher the score the more likely the account engages in targeted harassment, toxic trolling, or uses deceptive tactics engineered to cause division and chaos.”
They have four categories for accounts according to their scores: normal, questionable, disruptive, and problematic. Unfortunately, there is no API yet available for Bot Sentinel, so I had to scrape a sample of their data to perform my analysis. There are 303,069 disruptive or problematic accounts (scores ≥50%), and of all 26,992 I was able to get, 15,088 (55.8%) of them fell into those categories. I wanted to narrow the scope of my analysis a little bit more though, so I cut out all the accounts that had ≤143 followers (the median), and was finally left with 11,750 accounts. Doing this makes sure the accounts I’m looking at are really engaging with other users.
The Bots are Booming
Making a count of the number of accounts created each day since 2008, you can see a major boom at the very end of 2016 and start of 2017.
Finding Patterns
The charts below show two interesting patterns in disruptive bot creation: spikes on the first of each month, and spikes on in the fall of each calendar year.
Autumn: the time for bots
New Month, New Bot
Not only that, if we take another look at the counts for each day, we see the first of each month is an extraordinarily popular day to create a bot or troll account.
This pattern is so curious, it leads me to believe there is either some exceedingly good reason to launch a bot on the start of a given month, or perhaps there’s an error in the way the creation date is tagged by Bot Sentinel (perhaps, for example, a large number of bots were assigned a creation date of 10/1 if they were made anytime in October. I don’t have evidence for this).
What about 2016?
This was a historic election year where bots and trolls where accused of playing a major influence on the results, so I wanted to take a look at the growth of bot and troll accounts in the years leading up to the election.
There isn’t much of a difference until November 2016, and it was the 1st of the month. It isn’t likely that these specific bots and trolls had a huge influence on the results, within just a few days before the election. But looking at the changes between 2016 and 2019, nearly every month had over 100% increase of these bots and trolls. There is a chance that all the talk about bots after the election sparked an interest in creating more of them. Technology has gotten better since then, and there has also be an increased interest in political organizing — both left and right wing organizations.
Conclusions
Even though the dataset, which can be found here on Qri Cloud, is a relatively small sample of accounts on Bot Sentinel, the patterns here raise new questions:
- Why is the first of the month a time for new account creation?
- Even during non-election years, why is there a surge of new accounts made every day in the Fall?
- What happened in 2019 that lead to the increase of accounts we’re seeing now?
- Did the COVID-19 pandemic or the end of the primary elections cause a stop of account creation in 2020?
There is a possibility that these patterns are the result in a change of bot and troll identification, as well. Twitter itself has also been seeing a lot of growth (detailed here in their Q3 2019 Letter to Shareholders), although I’m not sure its enough to explain some of these patterns.
