Self-Improving AI Stirs

For decades, computers have followed precise directions in response to pre-defined events. That is, they’ve been programmed. Recently, some computers have begun to behave more flexibly after exposure to learning regimens. They’ve been trained.

But the next phase is upon us. Computers are beginning to develop and improve WITHOUT direct human involvement. The potential and implications are, of course, profound. Herewith, four serious examples.

Opposing Forces

Generative Adversarial Networks, introduced by Ian Goodfellow in 2014, feature pairs of AIs with opposing objectives. The “generator” creates data items and the “discriminator” assesses whether the items belong to a particular class.

For example, the generator might produce face images while the discriminator judges whether each image is of a real face or is a creation of the generator. The generator is rewarded for, and progressively learns to, successfully deceive the discriminator. To produce very convincing synthesized faces. Conversely, the discriminator is rewarded for, and learns to, successfully detect the generated from the authentic.

GANs have produced remarkable results in both synthesis and classification tasks, in some cases to superhuman levels, without long training times or large sets of training data. The opposing networks push each other to better performance.

Evolution

Neural network performance is achieved by adjusting the strengths of connections between neurons in response to experience. Learning, in other words. In December of 2017, Uber’s AI lab released several papers describing development of successful networks by, in essence, breeding rather than training.

Networks were designed, implemented, and tested. The best were recombined and mutated, and the process was repeated. No other form of learning was involved.

The approach produced state-of-the-art networks in maze-solving, playing Atari video games, and simulated human locomotion. In some cases, the evolutionary approach surpassed standard training methods.

Evolution With Learning

Also in 2017, Google announced AutoML. Designing a neural network to be effective at a particular task is difficult, time-consuming, and demanding in terms of required designer expertise. Google’s goal is to automate the task.

AutoML creates network designs, implements and tests the designs on their ability to learn the target task, selects the best performers, mutates them, and repeats the process.

Results have included not only state-of-the-art performance, but novel and interesting network structures in the process. Researchers are exploring evolved network design innovations.

Self-Play

For its famous defeat of the best human Go player, DeepMind employed very extensive databases of human games and practice play against human professionals. AlphaGo’s ability to absorb this knowledge effectively was celebrated.

Subsequently, however, DeepMind create AlphaZero, which learns entirely from self-play. Given only the game rules and the opportunity to play against itself, AlphaZero learned Go, Chess, and Shogi to world-best levels in very short periods of time. In fact, experts observed that the system often seemed to play not only better than, but also quite differently from, humans and from best prior available theory on optimum strategies.

In essence, AlphaZero constructed its own superior knowledge of the games, uncontaminated by human practices.

The Emergent Future

Each of these approaches allows developments that creators do not control nor, perhaps, even imagine. They are not without limits, of course, and all of these are cyberspace accomplishments only. But the underlying trend is clearly to harness competition, feedback, and continuous modification to produce ever more capable systems; as evolution always has. Outcomes are very difficult to predict.