“Safe” Artificial Intelligence

Next week Carnegie Mellon University is partnering with the White House Office of Science and Technology Policy to host a 2-day workshop on making AI “safe.” The first day is for “technical” white papers, which I take to mean “no humanities scholars need apply.” I don’t blame them—humanities research orients us to the largest goals we have as a society, and that can slow things down. As a language studies scholar, the question I would ask right now isn’t how we achieve safe AI; it’s “Who stands to gain and lose by talking about safety as a goal? What other options do we give up by prioritizing safety?” In other words, my assumption is that the way we talk about AI shapes (and is shaped by) the way we act on it.

Gains from the safety frame

The biggest gain to “framing” (that’s the technical term for it) the conversation in terms of safety, it seems to me, is that we begin to value some kind of oversight of AI research, especially in defining liability. After all, we only need safety when we have a risk of harm. When we have a risk of harm, we need mechanisms for ensuring people aren’t harmed (i.e. oversight, including training beforehand and accountability in the process and product of AI research), and we also need clear standards for what happens if people are harmed (i.e. definitions of liability). These outcomes are important and helpful, especially for increasingly autonomous systems — if a self-driving car accidentally kills someone, who is to blame? The manufacturer (e.g. Toyota licensing software from Google)? The system developers? The users (they shouldn’t have set it to go that fast anyway)? The car (as if it were a rabid dog that could be put down)? We see why the White House is involved in the safety frame for AI; oversight essentially consists of legal processes. DeepMind is the leader here, in requiring that Google institute an ethical review board as part of being acquired. Seeking oversight is especially proactive; review boards at universities weren’t required by law until the 70s when there was public fallout from heinously unethical government-funded medical research that had gone on for decades.

Who does the safety frame for AI benefit? Clearly, it benefits the workers who put it into place, the same way building codes benefit fire safety engineers.

Presumably, the standards that come out of a safety frame would also benefit end-users and others who interact with them. At CMU’s conference, this means protecting people’s physical safety, something like “Self-driving cars shouldn’t injure myself or anyone in my path.” With time, the safety frame could easily be extended to provide assurances of psychological/emotional safety (“Bots shouldn’t berate”); financial safety (“Stock bots shouldn’t bankrupt me”); identity safety (“Dating apps shouldn’t share flirting”); relational safety (“Personal care bots shouldn’t abandon me”), and more. If people’s safety is taken as a high enough priority, it could even be used to improve people’s safety. Should robots be mandatory reporters?

At a broader level, doomsday voices would argue that the standards that emerge from the safety frame could even save humanity.

Today, the safety frame is primarily being used to motivate researchers to join together. Thus, AI researchers (and companies that invest in AI) stand to benefit from the safety frame through favorable legislation. I think here of Amazon’s push for FAA regulations on drone safety. Insofar as those regulations end up conflicting with consumers’ desires, it is the corporations that stand to benefit from their collective lobbying. Consumer protections from AI will be more necessary as consumer exploitation becomes more tangible to people; this is a set of voices to be watchful and eager for.

So I should emphasize before continuing that there are important benefits to the safety frame. I’m not opposed to it in itself. We should just understand what it occludes, and not work with it exclusively (or even primarily). After all, any “frame” centers some things and pushes other things to the margins or outside the field of view.

Losses from the safety frame

Who stands to lose when we frame AI development in terms of safety? To answer that, we need to understand who we are being safe from. “Rogue AI” is the easy answer—no one wants to be exterminator by a paperclip maximizer! Still, trying to be safe from rogue AI casts AI as untrustworthy, and it works up people’s fear. As much as AI experts pooh-pooh depictions of robot uprisings in sci-fi movies, we participate in the sci-fi premise when we use the safety frame, making AI an “other” that stands against humans.

Us-them thinking on AI can stand in the way of AI integration into daily life. If AI is part of “them,” we harden our positive self-perception, judge AI more harshly, and generalize about AI on the basis of negative experiences.

Moreover, fear of AI is the seed for prejudice against it, which creates a representation problem. I won’t try to say here that AI can be wronged by prejudice, but I will point out that if people become prejudiced toward AI, it’s likely that AI makers will counter it by directing their products to be even more culturally mainstream: speaking standard English in a detached, unemotional way; using restrained gestures and little variation in tone of voice; speaking to get something across; dressing innocuously; having a light visual appearance... you know, being white and middle class. AI developers may not realize that their products have a culture, but once they do, they have to take on people’s pre-existing racial and cultural biases, and take responsibility for what they do with those. I think here of Samantha Finkelstein in Justine Cassell’s lab, who works on understanding how black children benefit educationally from talking with a digital peer who looks and speaks like they do. In my own research, I work with other white people as we try to see and overturn the ways white supremacy has penetrated our lives. Representation is important; AI research doesn’t need another us-them binary of humans versus AI to add to our fearful reception of others.

The harder answer to who we are trying to be safe from is that we are trying to be safe from the marginalized and oppressed. This exposes that the narrative “we” isn’t as unifying as it seems. “We” are the United States, conducting war recklessly against the Middle Eastern “them,” with robotics and computer science industries complicit: last year CMU, for instance, just received a $1.73 billion 5-year contract from the US Department of Defense to run the Software Engineering Institute. We are white Americans, mowing down black American “thems” with state-sponsored military gear, titillated by a presidential candidate who promises exclusion to brown “thems.” We are men, whose potential for accomplishments needs to be protected against women “thems.” We are cis-gendered, who need to be protected from transgendered “thems” in bathrooms. If AI development lingers in the frame of safety, it is easy for that safety to reinforce traditional “wes” who deserve safety at the expense of “thems” who have been seen as threats.

The safety frame, then, appeals to the fears of an in-group to create control. While there are some positive impacts to that, with AI, sitting in the safety frame could reinforce pre-existing biases, both in terms of the self-presentations of AI systems, and in the application of safety to other humans as they interact with smart systems.

Alternative frames

CMU’s workshop is one of four that the White House OSTP is sponsoring. So the safety frame is already complemented by the “governance” frame, the “social good” frame, and the “economic” frame. By analyzing any given frame, we see how it drives certain questions and releases others. Analyzing frames also serves to invent new ones. There could be the ecology frame — how will we interact with AI as part of an ecosystem? There’s the health frame — how do we interact with AI in a way that makes us grow? Most radically, there’s the justice frame — after all, we don’t call it artificial “intelligence” for nothing; while there are different standards for assessing what deserves moral consideration (e.g. in the animal rights literature, sentience, rationality, kinship), how can we ensure that future AI receive justice?