AI Risk: Should We Be Worried?
by Chief Research Officer at Neuromation, Sergey Nikolenko
Recently, discussions about the risk of “strong AI” have finally reached mainstream media. For a very long time, futurists and AI philosophers have been worried about superhuman artificial intelligence and how we could possibly make it safe for humanity to deal with a smarter “opponent”. But now, their positions have finally been heard by trendsetters among both researchers and industry giants: Bill Gates, Stephen Hawking, and Elon Musk have all recently warned against AI dangers. But should we be worried? Let us try to find out…
What is “strong AI”, anyway?
When people talk of “strong AI”, they usually define it rather vaguely, as “human-level AI” or “superhuman AI”. But this is not really a definition we can use, it merely begs the question of what “human level” is and how you define it. So what is “strong AI”? Can we at least see the goal before we try to achieve it?
The history of AI has already seen quite a few examples of “moving the goalposts”. For example — for quite a while the go-to example of a task that certainly requires “true intelligence” has been chess playing. René Descartes famously argued that no machine could be intelligent, an argument that actually led him to mind-body dualism. He posited that the “diversity” in a machine is limited by the “diversity” supplied by its designer, which early dualists had taken to imply that a chess playing machine could never outplay its designer.
Yet Deep Blue beat Kasparov in 1997, and humans are absolutely no match for modern chess engines. Perhaps even more significantly, recently AlphaZero, a reinforcement learning system based on deep neural networks, has taught itself to play chess by self-play, starting from scratch, with no additional information except the rules of the game; in a few hours AlphaZero exceeded the level of very best humans, and in a few days beat Stockfish, one of the best specialized chess engines in the world.
How do we, humans, respond to this? We say that early dualists were wrong and brush chess engines off: of course chess is a problem well suited for computers, it’s so discrete and well-defined! A chess engine is not “true AI” because we clearly understand how chess engines work and know that they are not capable of “general intelligence”, whatever that means.
What about computer vision, like recognizing other humans? That would require human level intelligence, wouldn’t it? Yet in 2014, Facebook claimed that it achieved human-level performance in face recognition, and this performance has only improved further since then. Our human response to this was to say that, of course, face recognition is not “true AI”, and we fall back on asking computers to pass the Turing test.
Alan Turing, by the way, was one of the first thinkers to boldly hypothesize that a machine would be able to play chess well. His test of general intelligence is based on understanding human language, arguably a much better candidate for a true test of general intelligence than chess or even face recognition. We are still far from creating a machine that would understand language and generate passable conversation. Yet I have a strong feeling that when a computer program does pass the Turing test, it will not be a program with general human-level intelligence, and all of us will quickly agree that the Turing test falls short of the goal and should not be used as a test for general intelligence.
To me this progression means that “human-level intelligence” is still a poorly defined concept. But for every specific task we seem to usually be able to achieve human level and often exceed it. The exception right now is natural language processing (including, yes, the Turing test): it seems to rely too intimately on a shared knowledge and understanding of the world around us, which computers cannot easily learn… yet.
Can we make strong AI, theoretically speaking?
Emphatically yes! Despite this difficulty with definitions, there are already billions of living proofs that human-level intelligence is possible regardless of how you define it. The proof is in all of us: if we can think with our physical brains, it means that our abilities can be at least replicated in a different physical system. You would have to be a mind-body dualist like Descartes to disagree with this. Moreover, our brains are very efficient, requiring about 20W to run, like a light bulb, so there is no physical constraint against achieving “true intelligence”.
Even better (or worse, depending on your outlook), we know of no principled reason why we humans cannot be much smarter than we are now. We could try to grow ourselves a larger cerebral cortex if not for two reasons: first, larger brains need a lot of energy that early humans simply would not be able to provide, and second, giving birth to babies with even larger heads would likely be too dangerous to be sustainable. Neither of these reasons applies to AI. So yes, I do believe that it is possible to achieve human-level intelligence and surpass it for AI, even though right now we are not certain what it means exactly.
On the other hand, I do not see how achieving human-level intelligence will make us “obsolete”. Machines with superhuman strength, agility, speed, or chess playing ability have not made us obsolete; they serve us and improve our lives, in a world that remains human-centric. A computer having superhuman intelligence does not immediately imply that it will have its own agenda, its own drives and desires that might contradict human intentions, in the same way as a bulldozer or a tank does not suddenly decide to go and kill humans even though it physically could. For example, modern reinforcement learning engines can learn to play computer games by looking at the screen… except for one thing: you have to explicitly tell the model what the score is, otherwise it won’t know what to optimize and what to strive for. And how do we avoid accidentally making a superhuman AI with an unconstrained goal to wipe out humanity… well, this is exactly what AI safety is all about.
Can we make it safe? And when will it hit us?
Elon Musk recently claimed that we only have a “five to 10 percent chance of success” in making AI safe. I do not know enough to argue with this estimate, but I would certainly argue that Elon Musk also cannot know enough to make estimates like this.
First, there is an easy and guaranteed way to make AI safe: we should simply stop all AI research and be satisfied with what we have right now. I will bet any money that modern neural networks will not suddenly wake up and decide to overthrow their human overlords — not without some very significant advances that so far can only come from humans.
This way, however, is all but closed. While we have seen in the past that humanity can agree to restrain itself from using its deadly inventions (we are neither dead nor living in a post-nuclear apocalyptic world, after all), we can hardly stop inventing them. And in the case of a superhuman AI, simply making it for the first time might be enough to release it on the world; the AI itself might take care of that. I strongly recommend the AI-Foom debate where Robin Hanson and Eliezer Yudkowsky argue about the likelihood of exactly this scenario.
On the other hand, while there is no way to stop people from inventing new AI techniques, it might well turn out that it is no easier to build a strong AI in your garage than a nuclear warhead. If you needed CERN level of international cooperation and funding to build a strong AI, I would feel quite safe, knowing that thousands of researchers have already given plenty of thought to inventing checks and balances to make the resulting AI as safe as possible.
We cannot know now which alternative is true, of course. But on balance, I remain more optimistic than Elon Musk on this one: I give significant probability to the scenario in which creating strong AI will be slow, gradual, and take a lot of time and resources.
Besides, I feel that there is a significant margin between creating human-level or even “slightly superhuman” AI and an AI that can independently tweak its own code and achieve singularity by itself without human help. After all, I don’t think I could improve myself much even if I could magically rewire the neurons in my brain — that would take much, much more computing power and intelligence than I have. So I think — better to say, I hope — that there will be a significant gap between strong AI and true singularity.
However, at present neither myself nor Elon Musk has any clue about what the future of AI will look like. In 10 years, the trends will look nothing like they do today. It would be like trying to predict at the year 1900 what the future of electricity would look like. Did you know that, for example, in the year 1900 more than a third of all cars were electric, and an electric car actually held the speed record in 1900?..
So should we be worried?
Although I do believe that the dangers of singularity and AI safety are real and must be addressed, I do not think that they are truly relevant right now.
I am not really sure that we can make meaningful progress towards singularity or towards the problem of making AI friendly right now. I feel that we are still lacking the necessary basic understanding and methodology to achieve serious results on strong AI, the AI alignment problem, and other related problems. My gut feeling is that while we can more or less ask the right questions about strong AI, we cannot really hope to produce useful answers right now.
This is still the realm of philosophy — that is to say, not yet the realm of science. Ancient Greek philosophers could ask questions like “what is the basic structure of nature”, and it seems striking that they did arrive at the idea of elementary particles, but their musings on these elementary particles can hardly inform modern particle physics. I think that we are at the ancient Greek stage of reasoning about strong AI right now.
On the other hand, while this is my honest opinion, I might be wrong. I sincerely endorse the Future of Humanity Institute, CSER (Centre for the Study of Existential Risk), MIRI (Machine Intelligence Research Institute), and other institutions that try to reason about the singularity and strong AI and try to start working on these problems right now. Just in case there is a chance to make real progress, we should definitely support the people who are passionate about making it.
To me, the most important danger of the current advancement of AI technologies is that there might be too much hype right now. The history of AI has already seen at least two major hype waves. In the late 1950’s, after Frank Rosenblatt introduced the first perceptron, The New York Times (hardly a sensational tabloid) wrote that “Perceptrons will be able to recognize people… and instantly translate speech in one language to speech or writing in another”. The first AI winter resulted when a large-scale machine translation project sponsored by the U.S. government failed utterly (we understand now that there was absolutely no way machine translation could have worked in the 1960’s), and the government withdrew most of its support for AI projects. The second hype wave came in the 1980’s, with similar promises and very similar results. Ironically, it was also centered around deep neural networks.
That is why I am not really worried about AI risk but more than a little worried about the current publicity around deep learning and artificial intelligence in general. I feel that the promises that this hype wave is making for us are going to be very hard to deliver on. And if we fail, it may result in another major disillusionment and the third AI winter, which might stifle further progress for decades to come. I hope my fears do not come true, and AI will continue to flourish even after some inevitable slowdowns and minor setbacks. It is my, pardon the pun, deep conviction that this way lies the best bet for a happy future for the whole of humanity, even if this bet is not a guarantee.
Sergey Nikolenko,
Chief Research Officer at Neuromation
