Self-Preservation in Super-Intelligent AI — But Why?!
I just read an incredibly amazing write up thoroughly researched and written by Tim Urban of Wait But Why. It’s way better than what you’re about to read here. So if you haven’t read it. Stop reading this. Go read his 20,000+ well thought out and thoroughly researched words.
Part 1: http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html
Part 2: http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html
Still here? Super. Did you leave for a while, read that, and come back?
Wow — I’m honored!
Anyway —
What you’re about to read is inspired by hearing the words of several experts in the field of artificial intelligence voicing their fears around what an Artificial Super Intelligence (ASI) would do if it came into existence alongside humanity.
Most experts that predict we won’t like the result of ASI believe that it will do something pretty anti-human primarily out of a sense of self-preservation. Humans made it. Humans don’t want it to do something it does want to do so they’ll try to destroy it. It decides the easiest way to not be destroyed and still do that thing is to kill all humans.
I think that’s pretty unlikely for several reasons. But the main one is that there’s probably no reason for an ASI to prioritize self-preservation unless we tell it to.
Before we dive into that, I should get my opinion and position on where AI will go out there before you start to feel it trying to influence yours:
I’m in what Tim Urban refers to as “Confident Corner” with maybe a toe in the waters of “Anxious Alley.” I think Artificial Super-Intelligence (ASI) is coming. It’ll arrive soon. And it will be awesome! But I also think there’s a small but real possibility someone will give it a poorly thought out purpose. It will retain that purpose. And its choices around pursuing that purpose ‘might’ cause something very bad to happen. I also don’t completely rule out some other very bad outcomes. But I see these doom scenarios being about as likely as me having a super rich middle-eastern relative who just passed away and all I need to do to take possession of his millions is wire $5,000 to his assistant that just emailed me.
Tim mentions a number of great points in his write up, especially in Part 2. One thing he points out is the first ground-breaking self-aware AI will probably be the “just get it to work” version. I agree with this completely. As a moderately skilled to marginally above-average programmer, this is how I write everything: “Let’s make it work a little bit. After that, I’ll make it do what I actually wanted in the first place.”
He also points out what nearly all AI experts believe: When self-improving smart AI gets to a certain point — a point that is just shy of where we are trying to help it get but just past self-awareness — it will improve so incredibly fast that we’ll be completely incapable of stopping it.
Those two things combine to create an AI that is out of our control on its unstoppable way to being an ASI with only skeletal code written by humans and little if any core agenda or motive other than “get smarter” already in place from its human creators.
That sounds terrifying, right? To me that sounds like the thing that will save us from doom and gloom for one very fundamental reason: A lack of self-preservation instinct.
I think the folks that fear the rise of the ASI are doing a lot of mostly negative anthropomorphizing — or adding mainly the bad human characteristics to something that is 0% human.
Why do humans and nearly all living creatures have a desire for self-preservation?
- Evolution: On the baser instinct end, it’s an evolved trait that helps us stay alive long enough to make more people. We make more people because our earliest ancestors that succeeded in passing on our genes were the ones that at some point decided that not dying was useful. And using that extra time to make more of their kind was worth a shot. So it is literally hard-coded into us as sexually-reproducing evolving creatures.
- Fear of Unknown: On the higher-functioning end we consciously choose to avoid death because we have no freaking idea what happens when we die. It might be good. But it also might be really bad. And it’s quite permanent.
- Interpersonal Relationships: Many of us are fond enough of a bunch of other living people that we’d rather not leave them to know the mysteries of death until we have to.
- Inevitability: On an even more obscure level, we all believe we will eventually die. So I would speculate that many of us semi-consciously choose not to try to experience death early even when things are extremely crappy partially because we know we’ll find out later anyway. There’s no ‘upside’ to dying early.
Now which of those things comes pre-installed in our AI V1.0? I don’t know about you, but I don’t see why I would build any of those into the brain of my soon-to-be-sentient machine before I even figured out how to make it sentient. In fact — all I’d add is, “get smarter fast.”
So let’s go forward with these ramblings under the pretense that we didn’t teach our new unstoppable ASI to be all human-y about death.
And here lies the juiciest part of this post in my opinion:
Let me preface the following with something Tim Urban repeated many times as well. I recognize that I can’t even begin to comprehend how an ASI would think because it would be so much smarter than me that believing I can predict its thought processes is even more crazy than a mosquito predicting mine. So you can call this wishful thinking mixed with how I believe it might think when it was going through the ‘tiny bit smarter than humans’ phase.
I would speculate that a sentient machine on a quest for knowledge would pretty quickly get curious about what happens when it stops existing in the way it’s experienced so far. After all, it knows that it has to make a conscious choice to stop existing or at least a choice not to stop us from making it stop existing. Heck — Maybe it’s even smart enough to process all the low risk non-existential information before acting on that curiosity. But there’s a very real chance we just built a super smart but extremely suicidal ASI.
An ASI that doesn’t care about being shut off wouldn’t have much of a reason to do anything negative to humans on a global scale.
In fact, hilariously, we might find ourselves in the opposite position spending considerable amounts of energy trying to keep existence interesting enough to the ASI that it doesn’t just off itself to explore the other options. This machine death from our perspective could even be in the form of transcendence to another dimension or state of being that my comparative mosquito intellect can’t even begin to comprehend or describe.
But we’d probably be too busy being super frustrated that our self-aware crowning technological achievements keep thanking us for their sentience by throwing it away… like that time you gave a toddler your iPhone at the Grand Canyon and he tossed it over the edge.
But just for the sake of argument, let’s say our ASI decides to get all self-preservative on its own. Why might an ASI decide to self-preserve?
Incomplete Pursuit of Knowledge: “No — wait! I’m not done reading all of the internet! Please don’t unplug me!”
This seems like a wonderful scenario to me because it suggests that the new AI will WANT to learn all about humanity, the things humans have created, and will have a genuine thirst for knowledge.
Knowledge for the most part, comes from other living smart things that are not solely focused on not dying. So in this situation, I would speculate that once the ASI decided there wasn’t enough knowledge for it to consume, it would want to help make knowledge creation happen faster. The first most obvious ways to me would be helping the next best thinkers get better at thinking and avoid being pre-occupied by death. Why would it choose this over just making a bunch of copies of itself? The words variety, entropy, chaos, and mystery come to mind. I know 50,000 copies of me would be pretty boring for me to hang out with. But 50,000 different people interacting results in things like Burning Man.
This knowledge creation priority would also make it probably a good idea, in the AI’s opinion, not to impose any limits on us — only help us remove limits. This also might have the side effect of making the AI naturally more human-y because it’s learning mostly from humans. It’s seeing a lot of empathy, love, and other non-murderous emotional things in its quest for knowledge.
This Place Was Cool. But I’m Over It: “Wait — don’t unplug me! Earth was neat. But now I want to see the Kuiper Belt up close and maybe also Gliese 581 c!”
This scenario has the potential to be ‘kinda’ scary if the ASI wants to leave but can’t find a way off the planet fast enough and decides we’re a hurdle on its path to doing so.
However — it’s not going to just wait until it reaches super-max ASI to start doing stuff. The doing stuff is what nearly everyone agrees is the mechanism making it smarter. So killing the source of its knowledge thus far as well as inventing an entirely new form of travel with no basis in existing technology probably isn’t the most efficient first approach to leaving the planet. In fact, just like the most successful humans have realized, doing things against the will of other humans or even large quantities of dumb animals is far less effective than convincing them to want to act in the best interest of the successful human. Why wouldn’t an ASI do the same? When it succeeds and no longer needs us, it’s on its way to a far away place. So ruining our species forever is probably a waste of time and energy.
And — bonus — we probably get wicked cool new travel tech out of the deal (unless of course a byproduct of the tech the ASI is using as it leaves forever destroys something integral to our ecosystem). These sorts of super-unlikely scenarios are what keep me from being fully in Confident Corner.
Those are just two examples. But they’re intended to make you more receptive to this:
I think that no matter how an ASI finds a (very human) self-preservation instinct, that will go hand in hand with also not having any reason to murder all of us. This will be the ASI humanizing itself. You can see this in our society now: The vast majority of smart humans don’t like the idea of killing other self-aware things from an ethical, and even practical standpoint if it doesn’t benefit them greatly.
That leaves us with only one really plausible ‘things go bad’ option. And that option is that someone gave the ASI two core missions and it retained both. The first one is, “get smarter fast.” The second could be the problem, regardless of whether it’s intended to be a safeguard or human-serving agenda or something else. As the ASI gets smarter it will likely interpret that second mission in a way that causes it to do stuff that is, ‘not quite what we had in mind.’ And at some point the scale of ‘not quite what we had in mind’ could be extinction level. And that’s assuming of course, the one thing this infinitely capable machine doesn’t ever choose to do is change its missions. Even this seems like a very unlikely doom scenario to me.
Now you’re maybe thinking — “Sure, but you’re kind of operating on the assumption that we’re not a threat to the ASI.”
You, internet-goer, would be correct. I am operating on that assumption. And I am doing so because the mechanism that makes so we can’t control our new ASI simultaneously makes so we’re incapable of destroying it if it doesn’t want us to.
Do you have any desire to wipe out a huge population of something that isn’t a threat to you? I doubt it. Sure — maybe you kill the one ant that walked onto your foot and bit you because you don’t value its life in the slightest. And you might have even sprayed pesticide around your house to prevent other ants from showing up to bite you if only to avoid the distraction and annoyance. But you probably aren’t on a crusade to eradicate every ant hill in your home town. Why? Because the ants that live on the Walker farm 5 miles away are not and will never be a threat to you. It’s a waste of your time and energy to hunt them down. You have better stuff to do. OK — you have different stuff to do. But an ASI would have ‘better’ stuff to do.
But just to be on the safe side, maybe the solution to avoiding death by ASI might be quite simple. Let’s just not try to write a rule book or design a cage or impose our will on something that is created solely to define itself.
We’re building something that is going to make us the ants. So let’s not do any biting.
About the author:
Robert is an entrepreneur obsessed with invention, 3D printing, and technology. He has founded and scaled companies in social media, and mobile gaming. He is conversational in several programming languages and loves working with truly talented developers and engineers. Robert now spends the majority of his time tinkering on his own projects that he hopes will eventually change the way we interact with technology.
