Neuroglia-The Alternative Model of the Brain
Walter Pitts, the other half of the duo that formulated the first model of an artificial neuron, burned his Ph.D. thesis and drunk himself to death. Why?
Ludwig Wittgenstein, after publishing his work to finally solve philosophy, one of the wealthiest men in Europe, gave up all his wealth and left philosophy at the age of 31. Why?
Both Pitts and Wittgenstein, the former who grew up impoverished and the latter who grew up wealthy, were gifted with superior analytical minds. Both investigate logical system only to come to an unassailable conclusion. Their formulations were wrong!
Are there problems that take years of intellectual effort to solve, or is most of the effort spent removing obstacles out of the way? Are there solutions that are simple but hidden by wrong assumptions? One could truly say that biology is hampered by the obstacle of lack of information about its intrinsic complexity. We don’t know what we are looking at so we can’t see the simple mechanisms. Here’s the rub though. We don’t even know if there are simple mechanisms!
Christof Koch of the Allen Institute of Neuroscience says that “don’t look to neuroscience for help with AI.” The brain is made up of the order of 1,000 different neuronal cell types that are linked to an order of 10,000 other cell types. Although we might understand how an individual neuron might work, we don’t understand how 10³ to 1¹¹ components interact. The brain is unimaginable complex and any hypothesis of how it works can be treated as relevant as any other hypothesis.
Neuroscience is primarily an experimental science. As a consequence, the prevalent bias is the search for a ‘micro canonical circuit’ that explains how the brain works. The alternative search to this, as demonstrated by the European blue brain project, is that the connectivity (dubbed connectome) will reveal the inner workings of thought.
From the perspective of Marr’s levels of analysis, neuroscience research is predominantly at the implementation level. This is in stark contrast to high-energy physics where experiments are motivated by existing theories developed by theoretical physicists. There are very few theoretical models of neuroscience. The lack of these models implies that experimental work is not focused to validate any of these models. Rather, neuroscience discovers new technologies to probe the brain and the results of these experiments drive new interpretations. This bottom-up approach has led to the fragmentation of knowledge. There are a lot of snippets of knowledge of how one subsystem might work but there are few grand unified theories. There are very few theoretical neuroscientists like Andy Clark:
Neuroscience is not the only way to study the mind. Philosophy, psychology and artificial intelligence have different doctrines in their approach. Any mixture of these approaches is sometimes dubbed as ‘cognitive science’. That is the transdisciplinary study of the human mind. But, I’m not sure if the label of a cognitive scientist is used often enough. Most researchers label themselves with the academic department they belong to. I suspect there are fewer cognitive science academic departments as compared to more traditional fields. It is difficult to survive in academia working on transdisciplinary areas. This is unfortunate because revolutions in science happen by breaking things and not just keeping things together.
In 1938, another genius, Alan Turing wrote in his Ph.D. dissertation: “we have gone to the opposite extreme and eliminated not intuition but ingenuity, and this in spite of the fact that our aim has been in much the same direction.”
In exploring Godel’s incompleteness theorem, he came to the prescient conclusion that a logical system (he used the word ingenuity) could be implemented by suitable automation, but he could not see how to do the same with intuition.
Intuition is a manifestation of an analog system. That is why an artificial intuition system like deep learning is based on the dynamics of a mathematical system formulated with continuous variables.
When the rubber hits the road or rather when a cognitive system presses itself on to reality (actually the reverse), it must do so through an analog system.
A digital system cannot feel an analog reality. At best, it approximates that analog reality. But it can only derive meaning and predictions if itself is analog in nature.
But all the neurons in the brain with its intermittent spike appears to be a discrete digital system. What then are we missing?
We are missing the fact that neurons are nothing but the information superhighway that connects the system. It is as if we analyzed the internet by looking only at the optical fiber and switches that connect it. Neurons are the wiring and routing agents of the brains. The internet of the brain. The hive mind in the mind. The true generating agents are of course the servers and devices that ring the internet. For the brain, these are the neuroglia.
The computational servers of the brain are these neuroglia cells called astrocytes (i.e. star cells). They have been conventionally been classified as glia. Just the white matter that serves as a kind of inactive glue for the brain. Consider that there are over 1,000 neuron types in the brain, yet there are just two types of astrocytes. Astrocytes are essentially the stem cells of the brain. They are what generate and manage neurons. It should be no surprise that the malleability of the human brain can be said to be a consequence of having more astrocytes as compared to other species.
Astrocytes can be thought of as playing a territorial game not quite unlike the game of Go. A person who loses his eyesight eventually learns to use their other senses in a more sophisticated manner. Essentially repurporsing existing unused capabilities to new uses. A blind person’s sense of hearing, touch, smell, and taste can be enhanced in a manner that can go beyond anyone with vision. The territorial nature of astrocytes implies that what is useful will continue to be useful and what is unused will be occupied to support what is useful. In software engineering, Conway’s law expresses that the organization of the software that is developed reflects the organization of the development team. In contrast, with brains, the structure of the human brain reflects the structure of what is used most often.
How did it come about that human brains have more astrocytes than other animals? I conjecture that it may be a consequence of a much richer umwelt. Umwelt is the idea proposed by Uexkull that an organisms cognition is influenced by what it can perceive and what its body can do. For example, crows with straight beaks know how to use tools that are straight. In contrast, keas with curved beaks only know how to use curved tools.
Humans have richer umwelts as compared to most animals. We have hands that are extremely dextrous with high sensitivity at the fingertips. We have eyes with extremely high resolution at our foveas. We have nimble jaws and vocal cords that allow us to express a multitude of sounds. Birds might have eyesight and vocalization at par of better than humans, but they lack the dexterity we have with our hands.
Now astrocytes may mirror the richness of our body plan in the same way that our brains mirror our body plan:
That is, where you do have neurons, you should find a lot more astrocytes. Said differently, like a city, where you find more roads you can assume you’ll find more generative agents (i.e. people). One can only speculate if Stephen Hawking, who lost most of his mobility, had a brain that could devote itself to a lot more abstract thought than the average person. All learning is a generative process. The more astrocytes you have devoted to new learning, the quicker that you will gain the competency.
The territorial game that is played by astrocytes can be framed as a social game. The model that I have proposed earlier, that of “selves and conversations” all the way up. Fits like a glove when we consider astrocytes as it implementation.
Framing this in terms of Marr’s 3 levels of analysis, we have astrocytes at the implementation level, social computation at the algorithmic level and homeostasis as the computational level.
It turns out that the tables are actually turned. It is the neurons that are the wiring and the processing centers can be found in the astrocytes. The entire field of neuroscience is named after the wiring and not the stuff that does the work!
Vertebrates (living things with a backbone) have evolved a complex adaptive immune system that allows their bodies to ward off pathogens that it has not previously encountered. In the vertebrate's adaptive immune system, b-cells are created in bone marrow and t-cells in the thymus. The invertebrate Octopus does not have an active immune system. However, octopus DNA is massive (33,000 protein-coding genes as compared to 25,000 for humans). Furthermore, these creatures are able to direct their genes, by ignoring or editing the gene's interpretation via RNA. For complex creatures, nature always seems to find a way to leverage an underlying digital code.
The method of vaccination is to teach our immune system to pre-recognize signatures of harmful pathogens like viruses. The immune system is a very sophisticated learning system that is able to remember previous infections over several decades.
Biology has the habit of using what it already has available. So whatever makes the vertebrate’s immune system adaptive is perhaps the same thing that the brain leverages to make it also adaptive. Why reinvent the wheel?
The way the immune system cells (T-cells) remember their targets is by replicating themselves. Memory is implemented by robustly creating multiple backups. It’s like storing information in a RAID drive. Memory in brains employs the same cloning mechanisms that our immune system employs to recognize pathogens. I conjecture, that the memory system of the brain makes use of similar mechanisms as the memory system of our immune system. It is not an analog system that is based on Hebbian learning but rather a digital system that stores information via the cloning of cells.
The conventional notion of neurons storing memory is more analogous to storing information in DRAM. The moment the power is turned of, then it’s all is gone! This doesn’t explain why we still are ourselves after we go under anesthesia. The sensation of going under anesthesia is very different from sleep. When you sleep there is still a feeling of time having passed, with anesthesia there is a complete blackout. Yet, we still wake up with our memories intact.
The adaptive immune system of vertebrates is able to scan the surfaces of infected cells to detect the presence of infection. It’s a sophisticated kind of pattern recognition system that is present in every t-cell.
Analogously, the astrocytes in human brains (that compose 90% of all the cells, only 10% are neurons) inherit similar functionality to provide the sophisticated pattern recognition required by brains. Studies have revealed that the percentage of astrocytes over neurons increases as a species becomes more intelligent. Chimpanzees' brains are composed of 80% astrocytes, other mammals have a lower percentage. Einstein's brain in fact has shown an unusual amount of astrocytes in certain regions. Research has shown that if human astrocytes are injected into mice, the mice become more intelligent.
Thus information is encoded with similar robustness of our immune systems that can be stored through several decades. Even as neurons in the brain are replaced through wear-and-tear, our memories remain due to their redundant replication.
Astrocytes are responsible for homeostatic behavior and as a consequence, this complex behavior is inherited to address the homeostatic requirements of higher-order cognitive selves. Antonio Damasio has argued that the purpose of the brain is homeostasis. Focusing then on astrocytes would further align with that notion.
We can derive the evolution of the brain from the initial condition of assuming homeostasis:
Memory, homeostasis and the self are all related functionalities. It makes more sense that they are implemented by the same rather than separate disparate mechanisms. A neuroglia formulation, therefore, unifies this under one mechanism.
The brain maintains several autonomous regions (i.e. selves) that are in constant negotiation with other-selves. The brain is massively modular and it maintains this modularity through constant competition between different regions. This kind of system allows the brain to maintain its adaptability to the world.
At a high level, they are not based on induction but rather based on a more sophisticated inference mechanism known as abduction.
Biological brains turn into jello if we don’t sleep. Maybe not jello, but it’ll feel like it turns to jello. Humans will die if they don’t sleep. It is the astrocytes that do the work to clean up all the junk that we’ve accumulated during the day.
Astrocytes are synchronized with the circadian rhythm to kick in and perform a lot of house cleaning. Not only are they physically removing the waste, but they are also reorganizing the brain’s information. Defragmentation of a hard drive may be a good analogy.
Essentially the brain’s information is indexical. That’s the purpose of the hippocampus. It’s a contextual indexing system that has rapid access to different memories.
The interesting thing about the hippocampus is that it generates new astrocytes. Not only is memory eventually stored, but an index is created. Why waste storing something that cannot be retrieved?
This indexing process happens not while we are awake but rather when we sleep. That’s when the astrocytes are decoupled from our sensor and motor neurons. This is when they can efficiently reorganize themselves into coherent clusters.
It is through this organization that stuff that might have an indexical similarity is connected to each other. That is how the analogy process that is the core of our thinking is enabled. We know that two things are related because it was previously stored as being related.
The amortized inference process is related but different from the process performed by artificial neural networks. They leverage the notion of code duality. Supporting the robustness of digital information as well as the power of analog computation.
DL networks are unable to create their own abstractions for a reason. This is because their inference process has hardwired the curves that require its fitting.
Biological brains are architected to perform a kind of top-down inference. In short, they see the whole before they see the parts. DL systems are implemented in the reverse. The parts are composed to see the whole.
But as the saying goes, if you focus only on the trees, you fail to see the forest. Well, that’s why DL networks can’t see the forest.
In the 1940s when cybernetics had its start, people had no answer to the questions about gestalt psychology. Why the hell do humans seek to complete the wholes?
This alternative formulation of how the brain works lead to a novel formulation of artificial neuroglia networks. Artificial neural networks have their origin in Walter Pitt’s model. It has been refined over the decades by Rosenblatt who made them continuous and Rumelhart who added backpropagation. But these are derived from Purkinje neurons of the Cerebellum and not the Pyramidal neurons of the Neocortex.
What would an artificial neuroglia or artificial astrocyte look like? Perhaps something that breaks through the semantic gap present between symbolic and connectionist architectures. Can we create new architectures using astrocytes as a model instead or Purkinje neurons?
I guess all of this is radical enough and so far from the mainstream that it just might work! After all, perhaps the reason we haven’t invented AGI is that we’ve all been looking in the wrong places!
