Understanding the human brain and where are we in terms of mimicking it in AI?
In my previous posts (here and here) I talked about the reason behind our obsession to mimic our brain. Now, lets talk about our brain, what it does, how much we know about its functioning and where we are in mimicking it.
Before I get your hopes too high, let me just state a humbling fact first…At this point in time, we don’t clearly understand exactly how human brain works. We know a lot of bits and pieces but we don’t have a “unified theory” i.e. a clear view of how all those pieces fit together.
Where are we in terms of our understanding of human brain?
The current state of affairs in brain science might be a little bit analogous to the time in cosmology and astrophysics when we had invented the telescope and had identified/observed few things but still hadn’t observed a whole lot and the a grand theory (like Einstein’s relativity) was missing. (though, as it turns out, quantum mechanics means that a ‘grander’ theory is still missing in astrophysics).
When it comes to human brain, with Functional MRI and other techniques we have the ‘telescope’ (actually ‘microscope’) but we don’t have a theory that explains everything that we observe + there are things that these techniques haven’t allowed us to observe yet. I will touch upon some of those in this post.
Anatomy & Structure Of The Brain — 3 important ideas:
In order to appreciate and understand how the brain works, its vital to understand the anatomy and structure of the brain. In order to keep it simple here, I will talk about 3 main ideas that are important for creating an AI i.e. mimicking the human brain.
- Different physical regions in the brain perform different functions
2. A neuron is the basic ‘building block’ of the brain. There are 100B neurons in human brain. Each neuron is connected to a set of other neurons making a “network” of an estimated 100 Trillion connections. Neurons transmit and receive chemical and electrical signals in somewhat of a binary format. They either “fire” or they don’t ¹ ².
3. The biggest idea that is relevant here is the “one learning algorithm” hypothesis
Apart from being the basic building block of the brain, a neuron also seems to be displaying a very interesting and important property which makes it likely that the same neuron can actually perform multiple different types of functions. This is a crucial discovery that has led to mimicking a neuron to create an AI that is capable of general intelligence without having to reprogram it for every different task. Since this hypothesis is so important for AI, I have added Andrew Ng’s two very insightful lectures in Stanford/Coursera (in the references section below) because they demonstrate the evidence behind this hypothesis in a clear and concise manner. I would highly encourage you to go through the text and/or video (#3 in the references section at the end of this post)
What does our brain do?
Our brain obviously does a lot of things, as you can see in the image above. Now even though I am not a neuroscientist, I have taken the liberty :) to put everything that the brain does into 9 key functions or ‘buckets’. I did this because it seemed logical to create a higher level abstraction of the details. This is interesting because it gives us some insight into the problem solving approach that the our brain takes when it needs to simplify things. Is there something here for our computers to take inspiration?
At the end of this section, we will get an appreciation of the complexity involved in trying to mimic the brain and state of affairs in AI.
9 Key Brain Functions And Where We Are In AI:
- Sensing (external): Visual information sent from retina of the eye to the brain, auditory information sent from cochlea of the ear, touch, smell, temperature of the environment (through skin) and so on..
- Sensing (Internal): Body temperature, water content, salt, glucose and oxygen levels in the blood etc..
- Motor control: Initiating body movements. For each and every organ in the body.
Again, I took the liberty to evaluate these 3 functions together when it comes to our understanding of brain and state of affairs in AI. I have done the same for several of the functions below where I have combined 2 or 3 functions for the sake of simplicity and it made sense to me in terms of logical abstraction.
Where do we stand with sensing and motor control?
Note that with Functional MRI and other techniques, we have observed what happens in the brain when we sense things in the external to the body and internally. We also have observed what happens when we control our body movements. So we know reasonably well the how and even the why? part of sensing and motor control.
So in essence, we have made massive progress in sensing and motor control of physical objects. For example we have motion sensors at home that can trigger an alarm and a software that alerts security agencies. In short, we are very good at sensing and controlling movement of things with machines.
4. Wakefulness and Sleep: This is called the circadian rhythm
Where do we stand with wakefulness and sleep?
We can observe what is happening in the brain during the circadian rhythm i.e. the difference between arousal and sleep states of the brain. Meaning that we know what happens in the brain when are we awake vs when we sleep. And we clearly know the bad things that happen when we don’t get enough sleep. We also know the good things that happen when we sleep enough including more creativity, memory, overall mental health etc. But we don’t yet exactly know why? There is likely more than one reason why we sleep? and hence multiple theories, some of which are getting good traction vs others. One of them being ‘garbage collection’ or cleaning up. Related topic of dreams is another area we know little about.
Its worth noting here that the circadian rhythm is likely to do a lot with the biological nature of this ‘computing device’. So its likely that we may not have bother about replicating this functionality while creating an AI. But its not impossible that we may realize that this circadian rhythm if applied to the AI also might help with efficiency, resource usage etc.
5. Learning, memory and decision making: This is the most exciting function of the brain when it comes to AI and we will shortly see why?
Where do we stand with learning, memory and decision making?
a. We know that we change our behavior and decisions based on experience so there is some mechanism in the brain that helps us learn all the way from when we are infants to the time until we die. There is clearly lot more learning during the initial years but learning as such, never stops.
b. In order to learn we obviously store information/experience (i.e. memory) in some form.
c. And using the learning and memory, we make decisions.
So combining these 3 into a single function made sense. You can view these 3 functions as the core of “computation” and by itself can provide us a big leap in AI if we can do these functions artificially in a way that is similar to the human brain. Hence this is where most of the focus is when it comes to AI research and applications. Self-driving cars and Natural Language Processing are two examples that I discussed in the previous post.
Lets talk about memory first. We know from experience and lot of observations that memory places a crucial role in learning and decision making process. There has been considerable advance in understanding how memories are represented and made in human brain, but we still don’t know a whole lot. Note that memories in today’s computers are vastly different compared to how the memory is created human brain. So its likely that we may have to re-architect the artificial memories in our machines to be able to replicate the human brain to achieve the speed in learning and decision making, in areas where our machines today don’t work well. Or maybe we can get away with the existing memory by only changing the architecture for learning and decision making. We don’t know yet.
Learning & Decision Making: This is where all the action in AI currently is!
When it comes to learning and decision making, there are 3 things we have observed in human brain that make it extremely compelling to replicate it to create an AI
a) Better ‘Algorithm’: We can do certain things lot better compared to the fastest computers today. Making decisions with limited and ambiguous information is a very important general capability because that’s how most decisions in the real world need to be made.
b) General Purposeness: What we have observed in the brain is that the nerve cells in each region of the brain (performing different functions) are very similar to each other in terms of their structure and how they work. And given that we know our brain can learn and do an extremely wide variety of tasks, brain is clearly a computing device that has an architecture which is suited for general intelligence. This makes it absolutely interesting and potentially very rewarding because as it stands today, the “intelligence” or architecture(hardware+software) in today’s computers is definitely not general. Meaning that every time we want the machine to do a different type of task, we have to write a different software. Hence its not so scalable.
c) Efficiency: Even if our brain consumes disproportionate amount of power (it consumes 20% of power with only 2–3 percent of body mass), it is still amazing that just a 3 pound object consuming less than 20W of power per day, can do things better than today’s super computers with massive amount of computing and power consumption. IBM’s Watson is a very good example of that. So we can learn immensely from brain when it comes to efficiency (doing more with limited computing and energy resource).
The field of neural networks is where most of the action, excitement and interest lies in AI. And this area is fundamentally focussed on mimicking the learning and decision making functions of our brain.
Now lets look at some of the ‘higher-order’ functions that human brain excels compared to other living organisms.
6. Creativity: Our ability to come-up with totally novel ideas from almost no-where is one of the fascinating things about human brain. Meaning without necessarily an exhaustive amount of computing and memory. We all know from our own experience that we sometimes come-up with new ideas just all of a sudden and many times by almost magically connecting the dots from extremely wide range of topics that may logically seem totally disparate and unconnected in any way. If we can figure out how our brain does that, then that could potentially help to solve some of the most intractable problems facing our society.
7. Motivation and goal setting: Even though motivation and goal setting may sound like a touchy feely topic, the reality is that goals drive our brain to do what we do. So goal is a fundamental aspect of why certain neurons in our fire in certain ways. A simple example is the goal to survive and live. So in the face of danger (like an approaching tiger) this goal translates to the ability of our brain to generate an intense flow of energy in the body that allows us to run at a speed that is way beyond our normal routine and stamina that we would generally think not possible to muster. Based on our observations inside the brain, goals are “encoded” in the form a variety of structures, chemistry and processes. For example, Amygdala is a region of the brain that is responsible for what is called as “Fight or Flight” response, which is needed for us to react to a dangerous situation. This region of the brain triggers a set of chemical processes that help us potentially to get a surge of energy in the body to run really fast in case our brain decides to opt for the flight response.
Its interesting to note that unlike hundreds of thousands of years ago when we were hunter gatherers, we don’t face predators in our daily life anymore. However our brain hasn’t necessarily changed dramatically to mirror that change in the environment. The likely reason is that the same Amygdala and its Flight and Fight response is now helping us from “modern-life dangers” like reacting fast during a likely accident scenario or fire etc. On the same lines non-physical dangers like bankruptcy i.e. concerns about finances, triggers the same area (Amygdala). So its fascinating to realize that even though the “hardware” of the brain did not substantially change over say tens of thousands of years, we were able to adapt to a completely new environment around us. This gives us an insight about the beautiful architecture of the brain when it comes to its “flexibility” or “general purposeness”.
8. Emotions: Closely related to motivation and goal settings, emotions are a fundamental aspect of being alive. Based on our observations of the brain and body, there are clearly neural activities and chemical processes associated with different types of emotions like happiness, love, anger etc. If we are creating AI that needs live with us and interact with us like humans do, emotions are a crucial aspect of those interactions. Hence we need to understand how and why emotions are created? And then mimic them in AI.
Where we stand with creativity, motivation and goal setting, And Emotions?
We know the different regions of the brain that gets activated (neural activity) when we do creative things, get motivated, set goals and become emotional. These are the ‘higher-order’ functions of the brain that are worth replicating if we truly want an AI who can do everything that we do and can “live” alongside with us. However, we still have a long way to go in terms of understanding how these functions in the brain work. Though one could argue that first order priority for us in terms of AI would be to first mimic the learning, memory and decision making functions. And then move into these higher-order functions as the second step because with the first order functions, there is little practical use of higher-order functions as such.
9. ‘Master control’: How are all the above 8 functions working in tandem? Is there a master control?
Where we stand with ‘master control’?
We have observed that even though the neural activity happens in different areas of the brain, each portion of the brain doesn’t function in isolation. So there is likely to be something in the brain that helps each of these brain regions and functions work in tandem (like a “network”). In other words some kind of a ‘master control’ or ability of the brain to ‘orchestrate’ all of the above functions. However we don’t know how this orchestration happens? And this is an important function of the brain that needs to be mimicked to make real solid advances in AI. In my later posts I will talk about certain approaches that are being taken in this direction.
10. Self-awareness: I said 9 things and tagged the ‘master control’ as the last one. But I am adding a 10th one because even though we experience self-awareness, we really don’t what it is? We don’t know if its something that is separate from the 9 things listed above or whether these 9 things when put together somehow automatically generate the idea of self-awareness? i.e We don’t if its really a standalone “function”. The other reason is of course that this gets into philosophy and religion, which is out of the scope of this blog! :)
- Here is a good video by Khan Academy about the basic anatomy of a neuron..
2. Here is another one that goes into a little more depth of how the neuron in our brain works…
3. The link to the videos of the lectures is below but I am putting down key excerpts from these lectures if you would rather just read it as text…
The brain can learn to see and process images, hear, learn to process our sense of touch, learn to do math, learn to do calculus, and the brain does so many different and amazing things. It seems like if you want to mimic the brain you have to write lots of different pieces of software to mimic all of these different fascinating, amazing things that the brain does. But there is this fascinating hypothesis that the way the brain does all of these different things is not with like a thousand different programs, but instead, the way the brain does it is with just a single learning algorithm. This is just a hypothesis but let me share with you some of the evidence for this.
This part of the brain, that little red part of the brain, is your auditory cortex and the way you’re understanding my voice now is your ear is taking the sound signal and routing the sound signal to your auditory cortex and that’s what’s allowing you to understand my words.
Neuroscientists have done the following fascinating experiments where you cut the wire from the ears to the auditory cortex and you re-wire,
in this case an animal’s brain, so that the signal from the eyes to the optic nerve eventually gets routed to the auditory cortex.
If you do this it turns out, the auditory cortex will learn to see. And this is in every single sense of the word “see” as we know it. So, if you do this to the animals, the animals can perform visual discrimination task and as they can look at images and make appropriate decisions based on the images and they’re doing it with that piece of brain tissue.
Here’s another example.
That red piece of brain tissue is your somatosensory cortex. That’s how you process your sense of touch. If you do a similar re-wiring process then the somatosensory cortex will learn to see. Because of this and other similar experiments, these are called neuro-rewiring experiments.
There’s this sense that if the same piece of physical brain tissue can process sight or sound or touch then maybe there is one learning algorithm that can process sight or sound or touch. And instead of needing to implement a thousand different programs or a thousand different algorithms to do, you know, to do the thousand wonderful things that the brain does, maybe what we need to do is figure out some approximation or to whatever the brain’s learning algorithm is and implement that and that the brain learned by itself how to process these different types of data.
To a surprisingly large extent, it seems as if we can plug in almost any sensor to almost any part of the brain and so, within the reason, the brain will learn to deal with it.
And there’s a sense that if we can figure out what the brain’s learning algorithm is, and, you know, implement it or implement some approximation to that algorithm on a computer, maybe that would be our best shot at, you know, making real progress towards the AI, the artificial intelligence dream of someday building truly intelligent machines.
Here is the full video that I will highly encourage you to see…