Review Validation: Automatic Filtering (RAF)

In the past year, several websites have come under scrutiny for how they have (or have not) validated and filtered online content. Youtube, for example, was criticised for their lack of consistency and transparency when removing extreme and defamatory videos. In its defense, the company blamed “overzealous moderators” for these inconsistencies and is now expanding its moderator workforce to over 10,000. Yet critics still argue that YouTube’s content removal is based more upon public outcry than the company’s stated policies on hate speech and violent extremism.

Yelp, on the other hand, is known for filtering its reviews based more upon user activity within their platform than on content. As a result, reviewers who search, post, and comment frequently show up on the front page, and new users are often filtered out, irrespective of what the reviews actually say. This strategy has led to a recent lawsuit that would have forced the company to remove a libelous review. Yelp won the lawsuit, and according to The Verge, celebrated the fact that it could not “be lawfully forced to remove third-party speech,” although others decried the ruling as “an invitation to spread falsehoods on the internet without consequence.”

The debate over free speech and accountability is not a new one, but there seems to be a growing belief that companies have a public responsibility to moderate their user content. Consumers are paying attention now more than ever to how algorithms choose which video auto-plays next, which reviews are listed first, and what submitted content is eventually filtered out. Now they are asking companies to pay attention too.

The Future of Review Validation Is Artificial Intelligence

How can companies that don’t have the resources of YouTube manage to determine what user-submitted content is legitimate? For Revain, artificial intelligence is the answer.

AI has been around for decades, but in the past few years, publications like Forbes, The Telegraph, and The Wall Street Journal have written about how it will revolutionise a broad swath of industries, from R&D to weather forecasting. By 2030, it is estimated to add as much as $15.7 trillion USD to the global economy.

Revain takes the responsibility of review validation seriously and envisions a world in which AI is used to make reviews more genuine, constructive, and transparent. In this spirit, they’ve designed an authentication system, which consists of two stages: automatic filtering and subsequent manual moderation.

The first stage, automatic filtration (RAF), utilises blockchain technology and artificial intelligence. It categorises reviews as spam, positive, or negative, based upon Natural Language Understanding and Tone Analyser applications of the IBM Watson AI. The company also applies proprietary custom filters and assesses account user history to determine if the review is legitimate.

One of the primary benefits of automatic filtration is that it does not require the involvement of third-party Revain employees. For smaller companies, an enormous workforce of moderators would be cost-prohibitive. Another benefit of the RAF system is that it utilises a modular architecture, which allows for versatility and customisation across a variety of industries. Lastly, although Revain does take into account user history to help avoid spam, it prioritises the content of a review when determining if it should move onto manual moderation.

The basis of Revain’s automatic filtration process is the IBM Watson artificial intelligence technology, which in recent years has transformed the business landscape for a variety of industries. But how does it work?

IBM Watson Artificial Intelligence

The IBM Watson AI was designed to interact with language not as humans would, but rather through a series of search and ranking systems to determine the most plausible interpretation and response to questions. Although its broader goal was to usher in a new era of technology that could analyse unstructured data, Watson was first tasked with competing on Jeopardy! In 2011, it went head-to-head with all-stars Ken Jennings and Brad Watson. And it even came out victorious.

Watson is unlike standard search engines, which rely upon matching keywords and often fails to answer complex questions — especially questions with as much subtlety and wordplay as the ones asked on Jeopardy! Instead, the question-and-answer computer was fed an enormous amount of data over a number of years. According to IBM, the cluster of computers that make up the AI can hold the equivalent of about one million books worth of information. When asked a question, more than 100 algorithms answer in different ways. Another algorithm scores and ranks each answer based upon the number of other sources that support or refute it. Approximately three seconds after a question is asked, Watson will offer the single most well-supported answer.

In January 2014, IBM launched a specific division to develop business applications for Watson, which is now used in a number of disparate industries, such as advertising, customer engagement, education, financial services, health, internet of things (IoT), media, talent, and work. Utilising deep industry-specific language and knowledge, the Watson AI can improve work processes and normalise varying sizes and types of data with minimal integration. Depending on the business necessity, Watson can also translate languages, recognise visuals, or offer insights into personalities. For Revain, the two most vital applications of Watson are Natural Language Understanding and Tone Analyser.

Natural Language Understanding and Tone Analyser

During the automatic filtration process, Watson’s Natural Language Understanding can analyse semantic roles, keywords, and relationships in a review and distill it to its intrinsic nature. That essence is then compared against learned patterns of human writing to determine whether it was written by a human or a machine. Considering the ability for bots to now inundate a website, the Natural Language Understanding AI can ensure that machine-written reviews are classified as spam.

Another prevalent review is the overly emotional one. According to the Harvard Business Review, online reviews tend to overrepresent the most polarised views, producing a “bi-modal” or “J-shaped” distribution with an abundance of extreme opinions and limited moderate ones. Reviewers who have a moderate experience might not find it worth their time and effort to leave a review.

Revain’s R-token incentive system counteracts the lack of motivation for moderate reviews, and extreme reviews can be detected by Watson’s Tone Analyser, which comprehends communication style, listens for social cues, and can predict which emotion a piece of text is evoking. Revain utilises the Tone Analyser to decide if a review is written with particular emotions — anger, disgust, or joy among others — that may be deemed malicious, excessively flattering, or otherwise unconstructive.

User History Analysis

User history analysis is a beneficial component of RAF, as it ensures that users are not acting in a malicious or irregular way. Geolocation analysis matches IP addresses of reviews and businesses to ensure that the individual is at or has been near the location of a physical business.

Revain also limits users to posting five reviews per day to deter accounts from spamming businesses and trying to take advantage of Revain’s incentive system. Lastly, Revain assesses users’ history for past automatic rejections. Users receive a warning if they receive three automatic or five manual rejections in a two-week period. The fourth warning leads to a ban from Revain’s platform and a withholding of any funds the reviewer may have earned.

Manual Moderation

If Revain’s automatic filtration deems a review spam, then the user will be penalised. If the review is determined to be legitimate and positive, then the user is incentivised. The only situation in which a company can review one of their user reviews through Revain is if the RAF determines that it is legitimate and negative.

During manual moderation, companies have the ability to reject reviews they believe are not constructive. However, by this point, a fragment of the review has been recorded in perpetuity by the blockchain. Therefore, it will still be visible to all users and cannot be hidden or revised. Even negative reviews in the Revain system are immutable and transparent.

Companies may also respond to negative reviews and argue why they are unconstructive, but if a user disputes rejection, the case will be arbitrated by a decentralised system of users with high reputations. If the user wins, the company receives a warning. The fourth warning will lead the company to be banned from the platform. This manual moderation system supplements and reinforces the legitimacy created by the automatic filtration system.

Conclusion

In the current climate of hoaxes and malicious reviews, consumers are developing higher expectations for companies in how they moderate and filter content. They don’t necessarily have the time or energy to wade through a sea of unhelpful reviews and want to be able to trust platforms that demonstrate transparency, consistency, and legitimacy.

For those companies, then, the question becomes how to validate user content without the back-end costs of an enormous team. Revain offers an automatic filtering system that is versatile and customisable, based upon AI technology that can offer you the solution.