Learning In Layers

How neural architectures memorize and generalize

Much of the power of artificial intelligence comes from copying natural mechanics of the brain. Neural networks learn by reshaping connections between artificial neurons across multiple layers based on the outer layers of the brain, the cerebral cortex.

These layers seem to be where much of our knowledge and memory live. When the brain is studied, scientists try to figure out which parts of the cortex are activated for certain kinds of thinking. It still isn’t well understood what is actually happening in those layers and how it really learns. It is nearly impossible to watch and analyze what’s happening in millions and billions of living neurons, but it is possible in artificial ones.

Neural networks in AI are set up with an input layer, an output layer, and various hidden layers in between. The input takes the raw set of data that the hidden layers use to extract meaningful features and identify or manipulate them in the output layer. As it gets exposed to more and more training data, each layer gets better at doing its job, making fewer and fewer mistakes. By reducing the error, it is said to be able to “fit” the data better.