Imitating Brain Plasticity in Deep Neural Networks

Neuroplasticity is one of the key mechanisms of the human brain. Can we recreate it in deep learning systems?

Jesus Rodriguez
Nov 11, 2020 · 5 min read
Image for post
Image for post
Source: https://twitter.com/lorishemek/status/1091703377799643137

I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

The human brain remains the biggest inspiration for the field of artificial intelligence(AI). Neuroscience-inspired methods are regularly looking to recreate some of the mechanisms of the human brain in AI models. Among those mechanisms, plasticity holds the key of many of the learning processes in the human brain? Can we recreate plasticity in AI agents?

Synaptic plasticity is one of those magical abilities of the human brain that has puzzled neuroscientists for decades. From the neuroscience standpoint, synaptic plasticity refers to the ability of connections between neurons (or synapses) to strengthen or weaken over time based on the brain activity. Very often, synaptic plasticity is associated with the famous Hebb’s rule: “Neurons that fire together wire together” which tries to summarize how the brain can form connections that last a long time based on its regular use on different cognitive tasks. Not surprisingly, synaptic plasticity is considered a fundamental building block of aspect such as long-term learning and memory.

In the artificial intelligence(AI) space, researchers have long tried to build mechanisms that simulate synaptic plasticity to improve the learning of neural networks. Recently, a team from Uber’s AI Lab published a research paper that proposes a meta-learning method called Differentiable Plasticity which imitates some of the synaptic plasticity dynamics to create neural networks that can learn from experience after their initial training.

The quest to simulate synaptic plasticity in AI models is nothing new but its relatively novel in the deep learning space. Typically, plasticity mechanisms have been constrained to the domain of evolutionary algorithms and specifically backpropagation techniques. The creative idea of Uber’s Differentiable Plasticity relies on extending backpropagation models with traditional gradient descent techniques to assign variable weights to connections between neurons in a computation graph.

Image for post
Image for post

Differentiable Plasticity Explained

The main idea behind Differentiable Plasticity is to assign each connection in a deep neural network an initial weight, as well as a coefficient that determines how plastic the connection is. A connection between any two neurons i and j has both a fixed component and a plastic component. The fixed part is just traditional connection weight wi,j . The plastic part is stored in a Hebbian trace Hebbi,j , which varies during a lifetime according to ongoing inputs and outputs. The algorithm evaluates the plasticity of a connection using a coefficient αi,j. Thus, at any time, the total, effective weight of the connection between neurons i and j is the sum of the baseline (fixed) weight wi,j , plus the Hebbian trace Hebbi,j multiplied by the plasticity coefficient αi,j .

Image for post
Image for post

As part of the training period, the Differentiable Plasticity model uses gradient descent to tune the structural parameters wi,j and αi,j, which determine how large the fixed and plastic components are. As a result, after this initial training, the agent can learn automatically from ongoing experience because the plastic component of each connection is adequately shaped by neural activity to store information. That model closely resembles the synaptic plasticity mechanisms in the human brain.

Differentiable Plasticity in Action

The Uber research team released an initial implementation of the Differentiable Plasticity model in GitHub. The code includes a series of experiments that evaluate the new meta-learning technique across several well-known scenarios. For instance, in an image reconstruction task, a deep neural network memorizes a set of natural images that it has never seen before; then one of these images is shown, but with one half of it erased, and the network must reconstruct the missing half from memory. This task has been notoriously challenging to solve by traditional networks with non-plastic connections (including state-of-the-art recurrent architectures such as LSTMs) but Differentiable Plasticity models seem to cruise through it.

Image for post
Image for post

Another test used to evaluate plastic network was the maze exploration challenge which has been considered one of the standard tests for reinforcement learning algorithms. In that task, an agent must discover, memorize, and repeatedly reach the location of a reward within a maze. During the test, the models using Differentiable Plasticity vastly outperformed traditional reinforcement learning models as shown in the following figure. As you can see, the model using Differentiable Plasticity(left) is able to navigate the maze way more efficiently than traditional reinforcement learning model(right).

Image for post
Image for post
Image for post
Image for post

The results of the Differentiable Plasticity experiments showed that this type of technique can become a viable model to improve the meta-learning abilities of deep learning models. Furthermore, the ideas behind Differentiable Plasticity can be immediately applicable to mainstream network architectures such as LSTM which can benefit from incorporating plastic connections. Obviously, Uber has plenty of reinforcement learning scenarios in which Differentiable Plasticity can make a difference in the long term.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Jesus Rodriguez

Written by

CEO of IntoTheBlock, Chief Scientist at Invector Labs, Guest lecturer at Columbia University, Angel Investor, Author, Speaker.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Jesus Rodriguez

Written by

CEO of IntoTheBlock, Chief Scientist at Invector Labs, Guest lecturer at Columbia University, Angel Investor, Author, Speaker.

The Startup

Medium's largest active publication, followed by +756K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store