Facebook’s Text Understanding AI Engine Eliminates Negative Comments
After months of development and in-house testing, Instagram recently released an AI filter meant to delete negative, offensive, or hateful comments. The system, which originally started with Facebook, is called DeepText and it uses context to help understand words, meaning, and possible intent.
Language is complex, even for humans, rife with multiple meanings, slang, and opportunities for misunderstanding, doubly so on the internet where social clues, punctuation, and larger context are often missing. Taken a word for word a comment like “Only whites should have rights” may seem harmless, but a full understanding of meaning and intent reveals hidden hostility. While on the reverse side a comment full of offensive words like “Fuck what, fuck whatever Y’all been wearing” loses it sting when you’re aware that they’re Kanye West lyrics.
Despite the difficulties, humans usually get this kind of understands correct where machines often are hilariously wrong. However, last year Facebook announced a breakthrough for machines interpreting meaning through context with their text classification engine, DeepText.
DeepText was developed around the concept of word embedding, a method which imitates the way language works in the human brain. It learns new words by attempting to understand the words around it first.
For example, the word white can change meaning depending on whether it’s near other words like House, power, snow, or Sox. Not only does DeepText understand meaning like a human, but it also gets better over time like a person does, too.
At Facebook, DeepText was developed to analyze the huge amount of user-generated text, organize it, and help their engineers develop products for users. Users talking about the White House might want to see more news related items in their feed. A user talking about white power may have his or her account flagged for review or possible deletion. A complaint about the White Sox should be recognized as concerning baseball, which should be further recognized as a sport and result in sport related content. And the word white used near snow could indicate a possible interest in winter gear unless you also mention other terms like dwarves and seven, meriting different content results.
Instagram, which has been owned by Facebook since 2012, instantly saw an opportunity to eliminate the spam that often clogged comment sections on the platform by using DeepText.
First, DeepText had to learn what to look for. A fleet of human Contractors was hired to sort through mountains of repetitive messages, sales pitches, and automated nonsense that the AI will be tasked to filter. Four-fifths of the data was given to the program, and then DeepText was tested on the final fifth to see how effectively it had learned what desired comments versus spam responses was.
By analyzing semantics and source of a message, DeepText was able to successfully determine human created content more times than not. For instance, a comment by a friend is more likely to be desired content than a comment from someone you don’t know, which increases the likelihood that a message could be spam. The algorithm was launched platform wide in October last year after Instagram execs were satisfied with the results, and spam began to vanish from feeds almost automatically.
Although staying tight-lipped about how much spam has been reduced or how precisely the tool operates, Instagram CEO Kevin Systrom has said how happy he is with DeepText’s operation.
So happy he was, he wanted to turn the DeepText service on to something more complicated but just as detrimental to a user’s experience: eliminating negative comments, specifically those that break Instagram’s Community Guidelines both directly or ‘in spirit’.
Again contractors sorted through about two million comments at least twice, deciding if it was within guidelines and classifying it into distinct categories if not (i.e., bullying, racism, sexual harassment, etc.take). At the same time, algorithms were being finessed and adjusted while the platform was tested on Instagram employees’ phones.
Just like with spam messages, the search for negative come takes into account the source’s relationship to the poster and semantics in context when rating a comment for negativity.
Recently launched, the system deletes negative comments although the original poster can tell no difference when looking at this content on his phone. While automatically set on for all feeds, the filter setting is optional and controlled by users through the settings menu.
Currently only available in English, other languages will be available soon for both comment filtering and spam deletion.
If you scroll through Instagram, you may still find negative comments, it’s that nature of online interaction and the imperfect combination of humans and AI. But given the challenges facing it, there are bound to be complications, especially when encountering words with multiple meanings.
Thomas Davidson, who helped develop a similar machine-learning system for hate speech identification on Twitter, gives examples of how difficult the challenges truly are. The following are innocuous tweets dinged as false positives with Twitter’s system:
“I didn’t buy any alcohol this weekend, and only bought 20 fags. Proud that I still have 40 quid tbh”
“Intended to get pics but didn’t have time.. Must be a mud race/event here this weekend… Is like a redneck convoy out there.”
“Alabama has overrated this yr the last two weeks has shown too many chinks in their armor WV gave them hell too.”
Instagram had no specific response to these sentences but did admit there could be errors made by the system, a combination of the original raters’ judgment and the AI’s data training. Ultimately a decision had to be made to make the system passive and possibly blocking too little to be useful, or being aggressive and blocking too many benign comments.
“It’s a classic problem,” Systrom said. “If you go for accuracy, you misclassify a bunch of stuff that was actually pretty good. So, you know, if you’re my friend and I’m just joking around with you, Instagram should let that through because you’re just joking around and I’m just giving you a hard time.… The thing we don’t want to do is have any instance where we block something that shouldn’t be blocked. The reality is it’s going to happen, so the question is: Is that margin of error worth it for all the really bad stuff that’s blocked?” He then added, “We’re not here to curb free speech. We’re not here to curb fun conversations between friends. But we are here to make sure we’re attacking the problem of bad comments on Instagram.”
If the system works ideally, Instagram could become the fuzziest and friendliest social media platform on the internet. But it could also come across as canned, artificial, and censured if taken to too far. Systrom is willing to take the chance. “The whole idea of machine learning is that it’s far better about understanding those nuances than any algorithm has in the past, or than any single human being could,” he says. “And I think what we have to do is figure out how to get into those gray areas and judge the performance of this algorithm over time to see if it actually improves things. Because, by the way, if it causes trouble and it doesn’t work, we’ll scrap it and start over with something new.”
Related Links;
More News to Read
