No, Killer Robots aren’t Secure

Joshua Foust claims drones need autonomy in order to defeat hackers.

In a brief article posted Oct. 8, Joshua Foust argued that lethal autonomy for robotic weapons is needed because remote-controlled drones are vulnerable to hacking. Noting that this was getting some play on the web, I called it the birth of a meme, and proceeded to show why it is wrong.

Foust replied first on Beacon Reader [paywalled] and again on his personal blog, as well as in a stream of tweets. While accusing me of arguing from science fiction, misrepresenting his position (he supports developing autonomous lethal robots), and being envious of him, he has avoided addressing the substance of my criticism.

Foust’s argument (the one I criticized) can be summarized, in his own words, as:

  1. “Drones have been hackable for years.”
  2. “[T]o make a drone truly secure… allow it to make its own decisions without a human controller: if it receives no outside commands, then it cannot be hacked (at least as easily).”

To which I reply:

  1. The vulnerability of military drones to hacking is overstated, not a major concern, and not what is driving the push to autonomy.
  2. Making drones autonomous would multiply the risks posed by the possibility of hacking, as well as by unintended software errors and hardware failures.

Point 2 is really the important one, but Foust’s only substantive reply is on point 1, so I’ll start there.

Drone hacking is really a phantom menace.

There are no publicly-known instances of successful hacking of US military drones, in the sense of gaining arbitrary control. It is relatively easy to make this almost impossible through air gapping, good security practices, and strong encryption of control link signals. I say “almost” because security lapses may occur, but this will bring us to point 2.

Foust points to the 2009 report that Iraqi insurgents were accessing video feeds from US drones, which were broadcasting those feeds unencrypted for the use of soldiers. But “TV pirating” isn’t hacking.

Foust also points to the fact that civilian GPS is vulnerable to “spoofing,” the creation of false signals which tell a GPS receiver it is in a different place than it really is. A 2010 report did claim that Iran had used this method to capture an American RQ-170 stealth drone, but few experts believe this; it is more likely that the drone simply malfunctioned. Military drones have access to highly encrypted GPS channels which use constantly-changing codes. These channels can be jammed with noise, blocking their reception, but military drones should then revert to inertial guidance and head home.

The closest thing to actual hacking of drone control systems that has been reported is the 2011 infection of ground computers at Creech Air Force Base with a keylogging virus known to be associated with games. Foust noted that this had led to “security concerns.” Which is true, but that a hacker might gain control of a weaponized drone through such a virus was not really one of the concerns. The virus probably crossed the air gap through a disk or thumb drive, and would have had no way to establish a live link back across the gap to an illicit operator.

In reply, Foust accused me of “faith in the unhackability of remote systems” and ignorance of “the real vulnerabilities built into drone technology and C2 [command and control] systems.” Well, first off, the only faith involved here is that C2 systems are governed by the laws of physics: malware can’t just fly across air gaps, so if you are rigorous about maintaining an air gap, you can’t be hacked. Also, we have strong cryptography based on rigorous math, and the only way even the NSA can defeat it is through end-runs such as hacking into your computer before you encrypt your message or after you decrypt it.

Now, Foust provides a link to the website of Jeffrey Carr, a computer security consultant who seems overly keen to credit Iranian claims, and even speculates that there may have been some connection between the Creech virus infection and the RQ-170 downing. But Carr posted all this in Dec. 2011, when the media was interested in the story. His haste at that time is indicated by his assertion that “The Washington Post has reported that Iran’s cyber warfare unit took over the controls” of the RQ-170; in fact, the WaPo story reported only that Iran had made such a claim, which “US officials” had “brushed off” as “preposterous.”

Carr also makes much of a USAF report on “Next-Generation” drones which somebody leaked at about the same time. It does contain some bland statements such as “Cyber threats represent a major challenge,” but doesn’t indicate that the challenge can’t be met. It notes, for example, that not all drone links today are strongly encrypted, and recommends standards for encryption to be used in the future.

I don’t find Carr’s speculations compelling, but that’s not really the point. I don’t claim that drones can never be hacked, just that the risk is exaggerated and not an intractable problem. If the military does screw up on security from time to time, they will discover and fix this.

More important to the present issue is that no matter how likely it is that drones can be hacked, moving toward autonomy will only make the problem worse. Much worse.

This gets to my point 2, to which Foust has not really replied at all.

Autonomy would only increase the risk of hacking.

Making drones capable of autonomous target acquisition and fire decision, as well as autonomous navigation, means providing potential hackers with the tools to cause much greater harm. Instead of simply spoofing GPS or otherwise interfering with controls and bringing down a drone, malicious code could then cause the drone to go rogue and attack anyone or anything. In addition, it would likely shut off the ability of legitimate operators to communicate with the drone and override its rogue programming. The only way to stop such a rogue killer robot would then be to destroy it, the old-fashioned way.

Note that it would also be possible for an unintended bug or hardware failure to cause similar mayhem. While one can imagine a conventional weapon getting stuck “on” and spraying bullets wildly, the possibilities expand vastly when a lethal robot has been given the ability to move around, select targets and decide to engage them all on its own.

Note I am assuming that even a drone with autonomous capabilities would still have links to human controllers, as well as ports through which it could be commanded or reprogrammed. This is consistent with the DoD’s very aggressive policy for killer robots, which seeks to introduce autonomous capabilities while also retaining capabilities for human control and “man-machine partnership.”

However, Foust’s argument (see his point 2 above) would require that autonomous weapons really be hermetically autonomous, not having any ports or links through which they could be commandeered. This presumably requires enough artificial intelligence for the weapon to recognize authority and understand its commands. The weapon must also be smart enough to resist “spoofing” in the form of false commands as well as mock-up tanks and other forms of tactical deception. Yet not too smart, lest it start thinking for itself.

I illustrated this last issue with a link to the hilarious scene from the cult movie Dark Star in which an astronaut tries to persuade a bomb not to detonate. This no doubt contributed to Foust’s rage at my invocation of science fiction to make a point.

I’ll just state for the record that I’m prepared to meet him online or in any forum for intellectually honest and depersonalized discussion.

And in any case, I’ll be back.

UPDATE: Foust finally provided a working link to my post in his post ostensibly answering it, and added an update at the end, which I suppose means we’re now having an ihadd via blog post updates.

In any case, his update cites a WaPo story on “al-Qaeda’s efforts to fight back against drones,” which mentions a number of things that are not hacking, including (Foust’s cite) jamming of GPS and control signals. Whether alQ can do this is doubtful*, but in principle it is possible; so is simply shooting drones down. The latter would be more effective, since the jammed drone would most likely, as I explain above, revert to preprogrammed flight on inertial guidance, and quickly pass out of the limited area within which the jamming might work. Plus, jammers make great beacons for missiles to home on.

UPDATE 2: Foust now claims that the argument I refute is not his, that he “reported it is the stance of the Department of Defense.” However, his original article simply does not say this; rather, it presents the argument in his own words (and I happily give him credit for it, not having ever heard or seen anyone else make it). Neither does his Beacon article, nor his own blog post (unless he’s updated it by the time you read this) disown the argument or attribute it to the DoD. Which is good, since the DoD has taken no such stance. Yet.

*GPS jamming to frustrate GPS guided missiles is certainly possible, and jammers are available on the grey market for this purpose. Whether they would be effective against drones flying at 5k-15k feet is more doubtful. Jamming of drone control signals would be even harder for non-state actors since the drones use highly directional dish antennas pointed upward at satellites, and can change frequencies if they encounter jamming.