Community Substandards of Facebook and YouTube
By Kathy Waters and Bryan Denny, Co-founders of Advocating Against Romance Scammers
Imagine breaking up with someone every day, sometimes more than once, for years. Imagine being months into a committed relationship with a wonderful partner only to discover the whole thing was fake. As cofounders of Advocating Against Romance Scams (AARS) we have heard thousands of stories like these, stories of the devastation wrought by romance scammers from the multiple types of victims they entrap: those whose identities are stolen to build fake profiles, those whose hearts are broken to steal their money, and those who are turned into money mules once their bank accounts have been wrung dry. These scams are psychologically and financially ruinous and they are happening every day, with the full complicity of Facebook and other social media and dating platforms.
According to the 2020 Internet Crime Complain Center Internet Crime Report, romance scams cost Americans more than $600 million last year, having quadrupled since 2019. Put another way, they are the #2 most lucrative ploy for scammers (business scams being #1). That should make stopping them #1 priority for both tech platforms and law enforcement. But because of Section 230 of the Communications Act, tech platforms have no liability to police user-generated content, and law enforcement has little recourse since scammers are often based outside the U.S. Most tech platforms have explicit policies that require them to remove any account that has stolen another person’s identity, or engaging in scams, but few platforms enforce these policies well.
Our most recent research project, Community Substandards: Capturing the Empty Promises of Big Tech’s Safety Against Online Romance Scams set out to test how effectively social media platforms are enforcing their own policies. After creating a fake profile, we uncovered and reported twenty-four groups on Facebook that were openly trading tips on how to romance scam and launder money, as well as where to buy fake profiles and scripts to lure victims in violation of Facebook’s community standards. We reported twelve videos to YouTube violating their policies by offering ‘How To’ guides to romance scamming. We reported active profiles demonstrably built from the stolen identities and photos of four of our members on Facebook. Collectively these men have seen their information used in more than 12,000 fake profiles netting more than $4 million for scammers. Two of the men’s profiles have been verified authentic by Facebook itself, so any others should automatically be considered fake.
The results were shocking: a month later, all the Facebook groups were still functioning, and even growing — one of them adding an astounding 50,000 more members. All the YouTube videos were still up and had increased in subscriber counts and views. Even more distressingly, Facebook determined that none of the fake profiles we reported broke their community standards, leaving them active to scam new victims. 0% of accounts that were demonstrably in violation of Facebook and YouTube’s own policies and the law were removed or disabled. Facebook’s one suggestion: that we block the offending accounts if we didn’t want to see them.
That appears indicative of big tech’s faulty approach towards this problem: ignore it and perhaps it will go away. But as the mushrooming sum of money scammed from Americans through online romance scams proves, ignoring this problem has caused the problem to grow exponentially. So why would Facebook and YouTube allow this to happen? Perhaps the recent revelations by the recent Facebook whistleblower that there were internal concerns about the contraction of their userbase and content generation offer a clue. As active users of multiple profiles and frequent participants in groups and chats, romance scammers are an antidote to both of these issues. Indeed, driven by the imperative to conceal their identity and play many different roles, criminal users appear more likely to boost Facebook’s user metrics than legitimate users.
Facebook offers scammers unbelievable opportunities on a silver platter: free office space (the platform), free raw materials, easy access to a global client base, and the ability to market and promote their work at no cost — and with no risk. However, if Facebook enforced its own standards, none of the accounts we found would be allowed to exist — even our research would have been impossible. And tech firms were regulated the same way as brick-and-mortar companies, rather than shielded by Section 230, the evidence AARS documented in this report would make Facebook and YouTube liable for facilitating and profiting off criminal activity. Instead, no one is held responsible.
Facebook and scammers get rich. Ordinary Americans pay, with their life savings, their hearts, and their hope. The chief of financial fraud for the FBI estimates only 1/3 of victims report romance scams because the fraud leaves people so ashamed and decimated. The only thing worse than the false affections of the scammers is the cruelly complicit inaction of the tech companies. They share the scammers guilt, responsibility, and profits.
It should not be those who are robbed that feel ashamed, it should be Facebook and YouTube and their boards and investors. We should not be alone calling for their accountability, we should be joined by Facebook’s advertisers, their users, and our legislators. Representative Adam Kinzinger has been a strong ally. We invite other members of congress to join him, and us in calling for real enforcement. It’s time to match their 0% compliance with 0% tolerance for their bad behavior.
Find the report in its entirety at advocatingforu.com
