Four Moderation Strategies To Keep the Trolls Away
To paraphrase the immortal Charles Dickens, who would have loved social media and online gaming:
It was the : ) of times, it was the : ( of times…
Today, our tale of two communities continues.
Yesterday, we tested our theory that toxicity can put a dent in your profits. We used our two fictional games AI Warzone and Trials of Serathian as an A/B test, and ran their theoretical financials through our mathematical formula to see how they performed.
And what were the results? The AI Warzone community flourished. With a little help from a powerful moderation strategy, they curbed toxicity and kept the trolls at bay. The community was healthy, and users stuck around.
Trials of Serathian paid the cost of doing nothing. As toxicity spread, user churn went up, and the company had to spend more and more on advertising to attract new users just to meet their growth target.
Today, we move from the hypothetical to the real. Do traditional techniques like crowdsourcing and muting actually work? Are there more effective strategies? And what does it mean to engineer a healthy community?
Charles Kettering famously said that “ A problem well stated is a problem half-solved”; so let’s start by defining a word that gets used a lot in the industry, but can mean very different things to different people: trolls.
What is a Troll?
We’re big fans of the Glove and Boots video Levels of Trolling.
The crux of the video is that trolling can be silly and ultimately harmless — like (most) pranks — or it can be malicious and abusive, especially when combined with anonymity.
When we talk about trolls, we refer to users who maliciously and persistently seek to ruin other users’ experiences.
Trolls are persistent. Their goal is to hurt the community. And unfortunately, traditional moderation techniques have inadvertently created a culture where trolls are empowered to become the loudest voices in the room.
Strategies That Aren’t Working
Many social networks and gaming companies— including Trials of Serathian —take a traditional approach to moderation. It follows a simple pattern: depend on your users to report everything, give users the power to mute, and let the trolls control the conversation.
Let’s take a look at each strategy to see where it falls short.
Crowdsourcing Everything
Crowdsourcing — depending on users to report toxic chat — is the most common moderation technique in the industry. As we’ll discover later, crowdsourcing is a valuable tool in your moderation arsenal. But it can’t be your only tool.
Let’s get real — chat happens in real time. So by relying on users to report abusive chat, aren’t you in effect allowing that abuse to continue? The damage is already done by the time the abusive player is finally banned, or the chat is removed. It’s already affected its intended victim.
Imagine if you approached software bugs the same way. You have QA testers for a reason — to find the big bugs. Would you release a game that was plagued with bugs? Would you expect your users to do the heavy lifting? Of course not.
Community is no different. There will always be bugs in our software, just as there will always be users who have a bad day, say something to get a rise out of a rival, or just plain forget the guidelines. Just like there will always be users who want to watch the world burn — the ones we call trolls. If you find and remove trolls without depending on the community to do it for you, you go a long way towards creating a healthier atmosphere.
You earn your audience’s trust — and by extension their loyalty — pretty quickly when you ship a solid, polished product. That’s as true of community as it is of gameplay.
If you’ve already decided that you won’t tolerate harassment, abuse, and hate speech in your community, why let it happen in the first place?
Muting Annoying Players
Muting is similar to crowdsourcing. Again, you’ve put all of the responsibility on your users to police abuse. In a healthy community, only about 1% of users are true trolls — players who are determined to upset the status quo and hurt the community. When left unmoderated, that number can rise to as much as 20%.
That means that the vast majority of users are impacted by the behavior of the few. So why would you ask good players to press mute every time they encounter toxic behavior? It’s a band-aid solution and doesn’t address the root of the problem.
It’s important that users have tools to report and mute other players. But they cannot be the only line of defense in the war on toxicity. It has to start with you.
Letting The Trolls Win
We’ve heard this argument a lot. “Why would I get rid of trolls? They’re our best users!” If trolls make up only 1% of your user base, why are you catering to a tiny minority?
Good users — the kind who spend money and spread the word among their friends — don’t put up with trolls. They leave, and they don’t come back.
Simon Fraser University’s Reddit study proved that a rise in toxicity always results in slower community growth. Remember our formula in yesterday’s post? The more users you lose, the more you need to acquire, and the smaller your profits.
Trust us — there is a better way.
Strategies That Work
Our fictional game AI Warzone took a new approach to community. They proactively moderated chat with the intention to shape a thriving, safe, and healthy community using cutting-edge techniques and the latest in artificial and human intelligence.
The following four strategies worked for AI Warzone — and luckily, they work in the real world too.
Knowing Community Resilience
One of the hardest things to achieve in games is balance. Developers spend tremendous amounts of time, money, and resources ensuring that no one dominant strategy defines gameplay. Both Trials of Serathian and AI Warzone spent a hefty chunk of development time preventing imbalance in their games.
The same concept can be applied to community dynamics. In products where tension and conflict are built into gameplay, doesn’t it make sense to ensure that your community isn’t constantly at each other’s throats? Some tension is good, but a community that is always at war can hardly sustain itself.
It all comes down to resilience — how much negativity can a community take before it collapses?
Without moderation, players in battle games like AI Warzone and Trials of Serathian are naturally inclined to acts — and words — of aggression. Unfortunately, that’s also true of social networks, comment sections, and forums.
The first step to building an effective moderation strategy is determining your community’s unique resilience level. Dividing content into quadrants can help:
- High Risk, High Frequency
- High Risk, Low Frequency
- Low Risk, High Frequency
- Low Risk, Low Frequency
Younger communities will always have a lower threshold for high risk chat. That means stricter community guidelines with a low tolerance for swearing, bullying, and other potentially dangerous activity.
The older the community gets, the stronger its resilience. An adult audience might be fine with swearing, as long as it isn’t directed at other users.
Once you know what your community can handle, it’s time to look closely at your userbase.
Dividing Users Based on Behavior
It’s tempting to think of users as just a collection of usernames and avatars, devoid of personality or human quirks. But the truth is that your community is made up of individuals, all with different behavior patterns.
You can divide this complex community into four categories based on behavior.
Let’s take a closer look at each risk group:
- Boundary testers: High risk, low frequency offenders. These players will log in and instantly see what they can get away with. They don’t start out as trolls — but they will upset your community balance if you let them get away with it.
- Trolls: High risk, high frequency offenders. As we’ve discussed, these players represent a real threat to your community’s health. They exist only to harass good players and drive them away.
- Average users/don’t worry: Low risk, low frequency offenders. These players usually follow community guidelines, but they have a bad day now and then. They might take their mood out on the rest of the community, mostly in a high-stress situation.
- Spammers: Low risk, high frequency offenders. Annoying and tenacious, but they pose a minor threat to the community.
Once you’ve divided your users into four groups, you can start figuring out how best to deal with them.
Taking Action Based on Behavior
Each of the four user groups should be treated differently. Spammers aren’t trolls. And players who drop an f-bomb during a heated argument aren’t as dangerous as players who frequently harass new users.
Filter and Ban Trolls
Your best option is to deal with trolls swiftly and surely. Filter their abusive chat, and ban their accounts if they don’t stop. Set up escalation queues for potentially dangerous content like rape threats, excessive bullying, and threats, then let your moderation team review them and take action.
Warn Boundary Testers
A combination of artificial intelligence and human intelligence works great for these users. Set up computer automation to warn and/or mute them in real time. If you show them that you’re serious about community guidelines early on, they are unlikely to re-offend.
Crowdsource Average Users
Crowdsourcing is ideal for this group. Content here is low risk and low frequency, so if a few users see it, it’s unlikely that the community will be harmed. Well-trained moderators can review reported content and take action on users if necessary.
Mute Spammers
There are a couple of options here. You can mute spammers and let them know they’ve been muted. Or, for a bit of fun try a stealth ban. Let them post away, blissfully unaware that no one in the room can see what they’re saying.
Combining Artificial and Human Intelligence
The final winning strategy? Artificial intelligence (AI) and computer automation are smarter, more advanced, and more powerful than they’ve ever been. Combine that with well-trained and thoughtful human teams, and you have the opportunity to bring moderation and community health to the next level.
A great real world example of this is Twitch. In December 2016 they introduced a new tool called AutoMod.
It allows individual streamers to select a unique resilience level for their own channel. On a scale of 1–4, streamers set their tolerance level for hate speech, bullying, sexual language, and profanity. AutoMod reviews and labels each message for the above topics. Based on the streamer’s chosen tolerance level, AutoMod holds the message back for moderators to review, then approve or reject.
Reactions to AutoMod were resoundingly positive:
Positive user responses and great press? We hope the industry is watching.
The Cost of Doing Nothing
So, what have Trials of Serathian and AI Warzone taught us? First, we really, really need someone to make these games. Like seriously. We’ll wait…
We learned that toxicity increases user churn, that traditional moderation techniques don’t work, and that community resilience is essential. We learned that trolls can impact profits in surprising ways.
In the end, there are three costs of doing nothing:
- Financial. Money matters.
- Brand. Reputation matters.
- Community. People matter.
Our fictional friends at AI Warzone found a way to keep the trolls away — and keep profits up. They carefully considered how to achieve community balance, and how to build resilience. They constructed a moderation strategy that divided users into four distinct groups and dealt with each group differently. They consistently reinforced community guidelines in real-time. And in the process, they proved to their community that a troll-free environment doesn’t diminish tension or competition. Quite the opposite — it keeps it alive and thriving.
Any community can use the four moderation strategies outlined here, whether it’s an online game, social sharing app, or comments section, and regardless of demographic. And as we’ve seen with Twitch’s AutoMod, communities are welcoming these strategies with open arms and open minds.
One final thought:
Think of toxicity as a computer virus. We know that online games and social networks attract trolls. And we know that if we go online without virus protection, we’re going to get a virus. It’s the nature of social products, and the reality of the internet. Would you deliberately put a virus on your computer, knowing what’s out there? Of course not. You would do everything in your power to protect your computer from infection.
By the same token, shouldn’t you do everything in your power to protect your community from infection?
Want more? Check out the rest of the series:
- The Other Reason You Should Care About Online Toxicity
- A Tale of Two Online Communities
- Doing The Math: Does Moderation Matter?
At Two Hat Security, we use Artificial Intelligence to protect online communities from high-risk content. Visit our website to learn more.
Just getting started? Growing communities deserve to be troll-free, too.