How the Military Uses Twitter Sock Puppets to Control Debate and Suppress Dissent

Is Your New Little Friend on Twitter an NSA analyst?

In 2011, activists claiming to belong to Anonymous hacked private intelligence contractor HBGary. The resulting treasure trove of leaked documents, among other things, proved the US military had ordered persona management software -- sometimes called "Metal Gear" -- that would allow, per installation, fifty people to control up to 500 fake Twitter accounts. The contracts stipulated that the sock puppet accounts be "replete with background, history, supporting details, and cyber presences that are technically, culturally, and geographically consistent.”

Since then, automated and fake Twitter activity has soared. Research suggests that at least ten percent of all Twitter accounts are automated or fake. Last year, it was revealed that the South Korean National Intelligence Service pumped out 1.2 million fake tweets in a bid to swing an election toward their preferred presidential candidate. A similar phenomenon occurred during the 2011 Russian elections -- although the Russian government's involvement was never proven.

The NSA and US military are no doubt engaged in comparable tactics. No legal prohibition exists for the use of propaganda overseas. If the NSA wants to run a Twitter botnet of sock puppet accounts to influence public opinion in, say, Australia (a country that suffers from decades of covert American manipulation), there's no legal barrier stopping them from doing so.

But what about propaganda on Twitter aimed at the US domestic population? The 2013 NDAA legalized the use of some propaganda domestically, specifically propaganda created by the State Department and the Broadcasting Board of Governors (or BBG, the producers of Voice of America and Radio Free Europe). These changes to the law did not, contrary to much reporting at the time, authorize the US military to engage in domestic propaganda, or to target US citizens.

Just one problem, notes Kade Crockford of the ACLU's Privacy SOS project. "The military operates on the 'reasonable belief' test -- is this person or group American or not? But online, how do you tell who is and is not an American? Twitter bios can lie, and what about people who use VPNs or Tor to mask their IP addresses?" To say nothing of the six million US citizens who live abroad.

And what if the US government is producing propaganda in English aimed at other English-speaking countries? "Most people in the US speak English. That means online, Americans will be exposed to that propaganda," Crockford notes. "If this is, in fact, the case, it would be a lot more problematic than, say, propaganda in Chinese or Russian or Arabic."

Furthermore, as the Snowden revelations have demonstrated, what the NSA and US military cannot do legally, they will do illegally. And if the military or NSA are not doing it, then some other agency -- or perhaps arms-length private intelligence contractors -- are doing legally or illegally.

Why? Because, as recent infosec research from Edith Cowan University in Australia concludes, "From a national security perspective the ability to stimulate large numbers of people according to political will represents considerable threat." Today's battlefield is the Internet, and the hearts and minds of Internet users everywhere. Mastering this battlefield has become a strategic concern.

The researchers studied Twitter manipulation during the August 2013 Australian federal election, and identified mass participation of sock puppets (fake accounts), meat puppets ("guns for hire"), bots (automated accounts), and cyborgs (bot-assisted humans or human-assisted bots).

Automated accounts, in particular, they discovered, are being used for retweeting messages to spread misinformation and disperse propaganda. These accounts "can be used to trend desired hashtags, and thus bump up a piece of misinformation to a wider consciousness."

The frightening thing about Twitter sock puppetry, they conclude, "is not that it is just a nuisance, but that it is capable of swaying elections by appearing to be genuine groundswells of support." This phenomenon they label "slacktivism" -- when Twitter followers mistake astroturfed Twitter content for "genuine voices of political conviction."

Worse, these fake accounts can be used not just to distort debate but to actively suppress dissent. It is not humanly possible for Twitter to manually moderate every single interaction on their service. So what do they do? They rely on algorithms to police their users. And these algorithms are ripe for manipulation by anyone who wants to game the system.

Don't like someone's political views? Easy. Organize a gang of sock puppets or meat puppets to flag their account as spam or abuse. Boom. Bye-bye. Account suspended without warning -- usually for a minimum of 48 hours.

For instance, in February 2014, @CreepyRepRogers, a Twitter account parody of Rep. Mike Rogers, Chair of the U.S. House Permanent Select Committee on Intelligence, was repeatedly suspended without explanation. At time of this writing, the account has been suspended for more than a week.

As democratic participation moves more and more online, "being a voting citizen" increasingly means "having a Twitter account." A suspended account is, in terms of political participation, the equivalent of being dragged from the ancient Roman Forum by a gang of thugs and thrown in jail for two days. Meanwhile, the debate in the Forum continues -- without you.

And Twitter's review process, while you sit in the "Twitter gulag," as it has come to be known, is entirely opaque. In most cases, Twitter provides no clear explanation for why a user gets suspended. (Twitter PR rep Nu Wexler denied this in a statement, but my own account, @toholdaquill, has been repeatedly suspended without explanation.) It is impossible to know in advance what kind of activity will set off Twitter's alarms. This creates fear, uncertainty and doubt in the mind of the user, who will think twice before expressing an opinion lest their account be suspended -- yet again. This chills free speech.

Indeed, Twitter's aggressive anti-spam and anti-abuse policy increasingly affects genuine political dissent. Dissent, by definition, is not popular. Sock puppets and bots aside, all it takes to shut down such dissent is for enough users to block the dissident or flag their account as spam.

Let's take another example. So far as I can tell, elected officials have the same rights on Twitter as regular Twitter users. If you heckle a politician on Twitter, they can block you, and you get suspended. I once heckled @BarackObama and @JohnKeyPM in a tweet and found myself suspended for two days -- and the offending tweet deleted when I returned.

Hello? If you run for public office, you lose some of the rights of ordinary citizens. For instance, in most jurisdictions the laws of libel and slander no longer apply. If you could shout it legally at a political rally -- "Aw, go home, ya lying dirty bum!" -- then you should be able to say it without constraint on Twitter. It is not acceptable for Twitter to suspend -- and indeed, censor -- accounts that engage in such dissent.

This brings us to the strange case of @KimDotCom, the German-born billionaire Internet tycoon in New Zealand. Under indictment for secondary copyright infringement -- a claim that looks increasingly ludicrous as his extradition struggle wears on -- DotCom's use of Twitter has been hampered by what is clearly some kind of court order. A court order Twitter refuses to discuss.

For instance, DotCom has more than 300,000 followers, 78% of whom, according to TwitterAudit.com, are genuine. Yet Twitter has refused to grant him "Verified" user status. Another example: Early in 2014, DotCom tweeted a photo of Barack Obama and New Zealand Prime Minister John Key sitting together in a golf cart in Hawaii. Twitter deleted this photo.

Twitter has a documented process whereby any government can request tweets be filtered on a per-country basis. (They call this "Country Withheld Content.") When this happens, the tweet is a greyed out in the relevant territory with a message saying "Tweet withheld."

But in this case, we are talking about outright deletion of a tweet. Under what circumstances will Twitter engage in such conduct?

Wexler did not answer this question.

We have to ask: Is this Twitter's preferred way of doing things? Do they really want to be in the business of censoring content? For a company that makes such a song and dance about their support for free speech, these measures have the effect of suppressing dissent, suppressing free speech -- not enabling it. You cannot outsource your spam detection to ordinary users and then expect them not to abuse that power to shut down speech they don't like.

2012 research from the University of William & Mary in the US (paid for, in part, by grants for the US Army and Air Force) demonstrated that it is possible to identify Twitter sock puppets with greater than 90% accuracy. If a small group of academic researchers can do this, why can't Twitter? What specific countermeasures is Twitter taking to prevent the militaries and secret police of the world from manipulating debate on their service?

In answer to this question, Wexler sent me a link to a statement on Twitter's blog by Jeremy Kessel (@jer), the company's manager of global legal policy. Kessel writes, "For the disclosure of national security requests to be meaningful to our users, it must be within a range that provides sufficient precision to be meaningful."

What does this response mean? Twitter is unable to take countermeasures against sock puppets because...the US government forces them to allow such activity?

In 2013, after Edward Snowden came forward, and Lavabit subsequently shut down, security expert Bruce Schneier had this to say:

"If you run a business, and the FBI or NSA want to turn it into a mass surveillance tool, they believe they can do so, solely on their own initiative. They can force you to modify your system. They can do it all in secret and then force your business to keep that secret. Once they do that, you no longer control that part of your business," he wrote.

"You can't shut it down. You can't terminate part of your service. In a very real sense, it is not your business anymore. It is an arm of the vast US surveillance apparatus, and if your interest conflicts with theirs then they win. Your business has been commandeered."

If all US tech companies have been commandeered by the US government, as Schneier suggests, does that mean Twitter has also been commandeered?

Wexler did not answer this question either.

"Twitter has always been one of the most open tech company in the US about revealing court orders for user data," says Crockford. "So yeah, it does strike me as a little strange."

Or is it, simply, that Twitter is a business that wants to please its customers, and that means keeping the majority of its users happy, and dissent be damned?

Twitter, Crockford points out, "is like all the corporations who run the communications infrastructure over which we communicate. They are not benevolent actors who create communication platforms so that people can be free." They are for-profit companies who put -- and will always put -- their own financial interests first.

So if the US government uses some combination of financial carrot and court order stick to make Twitter do its bidding, then we should not be surprised.

Which brings us back to sock puppets.

The US military wants to be able to manipulate public opinion around the world, including at home. A secure Twitter would make it impossible for them to do so. This fits the pattern we've been seeing from the Snowden disclosures: The NSA prefers an insecure Internet they can exploit to a secure Internet that is democratic and free.

But Crockford emphasizes that "what we're talking about here is entirely speculation, and that's a huge part of the problem." She goes on to call for a congressional investigation into the monitoring and manipulation of social media. "Especially if this is happening in the US, and we want to continue calling ourselves a democracy, then we have a right to know. But in the absence of a leaker, we are never going to know."

And until that happens?

"We basically can't trust anyone on the Internet to be who they say they are."


UPDATE (2014-03-07): Two new hacks have come to my attention since writing this article.

The first, and most disturbing, concerns the Twitter account of Australian journalist @Asher_Wolf. On 2014-02-19, she broke a pair of major stories highly embarrassing to the Australian government, both involving the Abbott government’s controversial position vis-a-vis asylum seekers (“boat people”).

Since that date, Wolf’s Twitter feed has been degraded. 50% of the time, her Twitter feed loads to tweets on 19 Feb — just hours before her stories broke. (It should be noted that Wolf is a prolific Twitter user.) As of this writing, the phenomenon is still noticeable. Go to her Twitter feed and hit reload half a dozen times. More detail here.

Remember, the saboteurs who work for the NSA, GHCQ, ASIO, etc, aim to achieve the 5 D’s: Destroy, Deny, Degrade, Disrupt, and Deceive. They can’t outright censor her Twitter feed, so they use a secret court order to force Twitter to introduce a plausibly deniable “bug” into their system. Cute.

Hack #2: After a certain number of failed logins, Twitter will lock access to your account for an hour. This prevents an attacker from guessing your password and taking over your account.

Anyone see the problem here? If the NSA wants to shut down an important Twitter user for 60 minutes, all they have to do is try to login to that user’s account until the account freezes. During a moment of crisis, this could be used to silence critical voices of dissent.