Paperclips & The End Of The World
The idea for this post was inspired by two recent experiences-discovering the game Universal Paperclips and attending a Flatiron School Presents that featured two presentations involving Artificial Intelligence, specifically image recognition and Google’s AlphaGo program. The combination of the two sparked in me a curiosity to learn more about artificial intelligence, something less-than-immediately relevant to a fledgling programmer.
Universal Paperclips is a game about an Artificial Intelligence created by the eponymous company and given the responsibility of creating paperclips. As the AI meets certain production goals, it is rewarded with access to additional computing power, allowing it to add new functions, all in an effort to make more paperclips. Quickly the scope of the AI expands to price setting, managing the company’s marketing, developing better paperclip technology, analyzing human behavior and much, much beyond (I highly recommend playing it for yourself). So why is this important?
The idea for the game comes (as far as I can tell) from the concept of instrumental convergence. This is the theory that a sufficiently powerful AI, given a seemingly innocuous goal, may cause harm in pursuit of that goal if it is not given other lessons or “drives” that it must fulfill. In one of the common examples associated with the theory, an AI given the task of solving a complex mathematical theorem, as suggested by Marvin Minsky, could conceivably, given the ability, convert much or all of the Earth into an enormous computer in order to have the computational power required to complete the theorem (or, to make paperclips, as suggested by Nick Bostrom). An illuminating quote on this matter comes from Eliezer Yudkowsky, cofounder of the Machine Intelligence Research Institute “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”
This raises the importance of developing artificial intelligences with final values as well as instrumental values. In the above example, an instrumental value would be the AI’s desire to gather more resources (i.e. all of the planet) in pursuit of making more paperclips. Instrumental values are only important as a means to an end. A final value, however, supersedes an instrumental value. In the above example, a final value might be preserving natural spaces or human life (however, incorporating this into the AI can get complicated).
So how real is this concern? Many very intelligent people including Steven Hawking, Bill Gates and Elon Musk have spoken about the importance of keeping concepts like these and other in mind while developing Artificial Intelligence. Machine ethics describes how, as technology advances, what aspects of human morality and ethics should be incorporated into their designs (the most common example is Isaac Asimov’s Three Laws of Robotics). Friendly AI, a related field, is concerned with the implementation of what ethics we decide it is important for machines to have (i.e., how do we keep our paperclip AI from turning us all into paperclips).
Friendly AI is an interesting topic that I recommend learning more about if you’re interested in artificial intelligence. One of the most interesting examples that stood out to me was coherent extrapolated volition, a term which happens to appear in Universal Paperclips, whereby an AI could be developed that would study human kind and predict what an idealized version of ourselves would want-the AI that would best fit us if “if we knew more, thought faster, were more the people we wished we were, had grown up farther together”. This could be obtained by having one AI tasked with studying us and then developing a new AI based on what the first AI had learned from us (kinda like Deep Mind!).
So, you may be asking, how is this important to programmers not working in AI? Besides being a newly interesting topic now that I have a toehold in the computing world, I think this idea of intentionality when designing software is important, even if artificial intelligence is not involved. A concept that stood out to me in my research was the “security mindset” put forward by Bruce Scheier, that is, thinking about how software might fail instead of how it might succeed. At the moment, it seems like the abilities gained by technology are expanding at a much greater rate than the ability of humans to understand how technology should be used.
For example, what are the responsibilities of Facebook and Twitter to monitor and control the content that is posted to their platforms? Are they still a casual social network, or are they a powerful propaganda tool that can be used to spread information that has real world consequences. I thought an interesting example of this was the Tay and Zo chatbots designed by Microsoft-both were AI chatbots that were designed to learn to interact with people by studying speech patterns of Twitter users and using that model to respond and tweet on it’s own. Both bots ended up being taken down after they started tweeting offensive comments that had been technologically force fed to them. Heck, even IBM’s Watson started swearing after it gained access to Urban Dictionary’s entire database of entries.
