Staying Safe While Monitoring Extremist Groups on Fringe Platforms

The events of January 6 have changed the content moderation space.

Even for those of us who who investigate misinformation, the past couple of weeks has been a whirlwind.

After The Great Deplatforming, we are left with more questions than ever before. There is the question of moderation up and down the tech stack, as Joan Donovan has been writing about for years.

There is the question as to whether this actually is unprecedented; platforms have taken actions like this outside the Anglosphere before, but it’s received way less attention. That is one of the many points Jillian C York made after compiling a list of everything pundits are getting wrong about this moment in platform moderation.

There is obviously the question of free speech and moderation itself. Now it seems that every platform hosting content, big or small, has to think carefully about and act on these discussions.

Finally there is the Great Migration of MAGA supporters to alternative platforms. This has become very tricky for disinformation investigations.

Since January 6 we’ve had a number of questions coming from clients asking for help on monitoring alternative platforms like Telegram, 8kun, Gab, Discord, etc.

Our response has been very deliberate. We want to be careful in thrusting people into these platforms where there is a real danger of them being identified, harassed and threatened.

Our primary advice is to reach out to experienced partners like Kinzen, First Draft, the Institute for Strategic Dialogue, or any others in this space that you feel comfortable with.

If you are stepping into monitoring these platforms, please keep these rules in mind:

• Don’t use your real name, username or profile picture. Don’t post anything that could in any way lead to you or your family being identified.
• Don’t use your regular email address and password when accessing extremist platforms. Assume everything you do on sign up could be made public. Create a “throwaway” email address if needs be.
• Don’t reuse passwords across platforms. There is a chance you will be identified with this information.
• If at all possible, use a different phone number to your own. There are big privacy questions with some of these alternative platforms, and the last thing you need is neo-Nazis calling you up at the weekend. If that’s not possible, you can hide certain information on Telegram and other platforms.
• Consider using a VPN which helps with anonymity.
• Check out some self-doxing guides like this one from Access Now, this one from the Global Investigative Journalism Network and this one from The New York Times.
• Don’t engage in conversations with these extremist communities. You are stepping into a world of white supremacists, militia and terrorism.
• You might consider screenshotting material for posterity. Often, content is deleted by uploaders and you won’t be able to back up your points without evidence.
• Be careful believing everything you see. Alas, with less moderation and verification, there are plenty of fake accounts for prominent actors.
• Watch out for your mental health. Spending time with extremists in less moderated spaces will take a toll. Check out advice from the Dart Center.

Aside from security concerns, remember that a lot of the chatter in these closed networks is self-referential. It might take a lot of study before you “get” what is going on. Much of the conversation will be attempts at childish gags or contain lewd/violent imagery. Much of the content will require further verification before you can think about acting on the information you’ve seen.

Our job at Kinzen is to support moderators and researchers, and indeed any organisation at risk from disinformation. We’ve set up a daily briefing which helps navigate a rapidly evolving landscape. It covers the latest disinformation narratives and the key content that is promoted every day.

To get it, you can get in touch with us at partnerships@kinzen.com.