Why does AI still suck and can it suck less?
[epistemic status: I’m an amateur researcher. This article is partly me explaining the challenges of my own domain to myself.]
I recently got an email from a student asking why Artificial Intelligences (AIs) aren’t regularly deployed and tested in complex environments. This made me think about how best to phrase the limitations and challenges that AI currently faces. The short answer is that AI stills sucks because it’s hard. The long answer I’ve tried to divide and conquer below.
Planning is super hard.
Using imagination and limiting what kind of computations a person can/should do in their head is super hard. Especially in an environment that’s constantly changing. Look at what stuff Atari Deep Mind failed on. Anything that required some degree of planning or modeling of the environment (Asteroids, Montezuma’s Revenge, Private Eye… etc.), aside from basic short-term reactions to states, were utter failures.
Even though AlphaGo, also from Deep Mind, managed to succeed in a game with extensive planning, this becomes a lot less impressive when you realize that for it to gain a map of the game, it required 1202 CPUs and 176 GPUs and a specialized learning architecture to tackle the problem.
Knowledge representation is super hard.
Presumably, Atari Deep Mind could have the concepts of how asteroids explode, spacial navigation and inductive reasoning, programmed into it via examples. However, we then have the problem of how these skills relate to each other and how they can be combined. Even worse, figuring out depending on context where/when they should be deployed. Knowledge is hard to represent and it gets even harder when you have to figure out in a world filled with sensory information, what knowledge cues to pay attention to.
Scaling skills acquired is super hard.
For the sake of argument, let’s assume that thanks to some clever learning technique, Atari Deep Mind was able to learn how Asteroids explode and plan around them. It can figure out based on cues when it should be modelling asteroids. This is amazing, but if we switch the game from Asteroids to Minecraft we’re going to be right back where we started, with a set of skills that don’t apply.
For a real AI, getting good at one game should mean that you can transfer the skill to another game. The basics of momentum demonstrated in Asteroids, should be able to be applied or at least explored in the context of Minecraft. Just like the skills of hammering and nailing when applied to building a table should be able to be applied to putting up a window. But unless a hierarchy of skills is developed, with individual atomic skills at the bottom waiting to be redeployed to accomplish higher level goals, knowledge transfer is impossible. And don’t even get me started about learning to develop a hierarchy automatically!
So machines all have really hard times with the aforementioned problems of scaling, planning and knowledge representation. You could even argue that these problems are all rephrasing the same underlying enigma. But you know who’s pretty good at this stuff? Humans.
Taking inspiration from biology, is also super hard, but seems promising.
Often when I first introduce people to what the lab I belong to works on, their first reaction is to try and compare us with other approaches to Machine Learning via our performance on certain benchmark tasks.
This isn't the greatest comparison, because better performance on specialized tasks (such as question answering, image recognition and translation) is not a good reason to focus on biological plausibility. Going towards human intelligence by obeying biological constraints, taking hints from the cognitive abilities of humans and using the architectural benefits of biological substrates, is a legitimate reason.
Is this the path that’s guaranteed to bring us the glory of general intelligence? I have no idea, but to me it currently seems like the best bet.
