Turing was wrong
But it was actually Golgi’s fault
In 1948, Alan Turing wrote
“the cortex of the infant is an unorganized machine […]
made up from a large number N of similar units […]
connected to a central synchronising unit.”
This is wrong. For 2 reasons. (1) There’s no centralized synchronization unit in the brain, and (2) the brain is not just a set of connected wires. While the first is widely accepted, the second is less known, fueling a simplistic view of neural systems that facilitates and limits research and development. To Turing’s defense, these were not the main concepts he was trying to make, and reflect the knowledge of the times.
Out of sight, out of mind.
Turing’s view of the brain was informed by Camillo Golgi’s discoveries: He was the first to show that the brain is made of a network of wire-like structures. This wasn’t a trivial achievement because the brain, even under a microscope, appears as an unstructured whitish mass. The Italian physician achieved this breakthrough by developing a new visualisation method —Golgi’s method: he treated the brain with a chemical compound that he developed, and —lo and behold— only neurons would absorb its black color, making the indistinct surrounding matter disappear to the eye.
Out of sight, out of mind: The surrounding matter almost vanished from intellectual activity, as the attention focused on the intricate wiring. Santiago Ramón y Cajal’s improved staining of the neurons showed even more fascinating detail, and led to the “neuron doctrine”, in which the surrounding matter does not appear. After all, it was the century of electricity, the new force that would brighten us all — outside and inside. The wire-like structures of the brain created a new worldview that made neuroscience as we know it today, with connectionism as the dominating standard model.
It’s tempting to think that the matter surrounding neurons serves just as mechanical and biological support. It keeps them in place and well fed. But there’s more: The matter surrounding neurons in the cortex is conductive. This means that whatever happens in the neural cells, is reflected in the surrounding medium (unless there’s a thick, isolating myelin sheat, as in the white matter. It will be interesting to see how signals are propagated from and to the Nodes of Ranvier). If there’s a current spike inside, it also propagates outside. If a coded signal transits inside, it propagates like a wave outside (this is the basis of EEG). Viceversa, what happens outside the neuron affects what happens inside. If a coded signal transits outside, the signal might be picked up and reflected inside the neuron.
Now, the actual biophysics of these phenomenons are very complex. The waves travel through an heterogeneous medium, with varying impedances, sinks, and sources. A signal can be induced from an external wave into a neuron through electrical coupling and exchange of ions (ephaptic coupling) or by influencing the electrical synapses. Modelling these phenomena is extremely difficult. Nonetheless, the fact that the matter surrounding neurons is indeed a communication medium is remarkable.
Follow the tingling
If information travels through matter and not only through “wires”, neural systems would have 2 transmission modes. This would represent an ocean of largely uncharted mechanisms and dynamics, ready to be discovered. Every true explorer, right now, is feeling a little tingling. Of course, there are reasons for which extracellular communication might not play a role in information transmission. For instance, the intensity is too weak. Or the environment too noisy. An alternative purpose is thought to be signal synchronisation. But let’s briefly follow the tingling to see where it leads us to and what we can learn from the journey.
There are a few fundamental aspects that we know are true:
- The intracellular mode is spatially precise: the signal follows the wire down its branches. The spatial configuration —the topology— of the adult wiring changes slowly, mostly it doesn’t change at all.
- The extracellular mode is spatially coarse: the signal propagates freely in a volume, with sources and sinks broadly determining its direction. This cannot be replicated by wires because it is a continuum.
- We also know that the intracellular mode propagates strong, fast spikes of current, while the other mode is weaker and slower.
- There should be an interface between the 2 modes, otherwise why let the current leak? If all that matters is timing, it'd be better to isolate everything with myelin.
(1) and (2) indicate that the two communication modes would be somehow complementary, with extracellular excitation or inhibition providing a sort of common ground, a context. Additionally, while the intracellular topology is rather fixed, the spatial influence of the extracellular communication can be changed by the system in real-time. In other words, the activation patterns of the wired connections can temporarily modify the extracellular currents.
Then, (3) indicates that to achieve the same effect as intracellular communication, extracellular waves must act together, population-wise. On the other hand, local extracellular waves can achieve a modulation of intracellular activity.
The last piece, (4) is the most intriguing. An interface between a fast and a slow system is another sign of modulation. The real kicker however lies in how the interaction might occur.
Waves move through the ocean, and finally crash on the beach. Sitting on my surfboard outside the line-up, however, I don’t have the impression of horizontal motion, only a gentle up and down. This condition of planar static observer allows me, once at the top of the wave, to look off-shore for other peaks, and determine whether a good set is coming in. If there’s enough space between me and the next peak, I have sufficient time to move to the appropriate position …and start paddling for my life!
The spatio-temporal nature of waves travelling at the same speed allows transforming the observation from time to space, and viceversa. You can freeze the image of a wave and look at its spatial shape or stand at a specific point and see how the height of the wave changes over time. For a sine, the first gives the spatial wavelength, and the second the temporal period. Their ratio is the speed of any point of the wave.
When an extracellular wave hits a dendrite, the same phenomenon occurs: Temporal sequences can be read by sampling different positions in space.
It is a simple concept, but it allows for (1) Information Communication and (2) Learning.
- Waves are created by trains of spikes. The sequence is picked up by a dendrite having active locations at the corresponding spatial locations.
- The continued stimulation of dendrites at specific locations promotes the creation of active locations.
The communication principle of spatio-temporal coding is supported by the existence of another neural system using a similar version: the cochlea. Sound waves propagate through the inner ear, where different locations sample different frequencies.
While capturing specific sequences is easy, creating them requires fine adjustment of extracellular current sources and sinks. At our company, in order to create applications based on these principles, we became fluent in both emission and reception. The hardest part we achieved was conciliating the connectionist paradigm with extracellular communication in a unique, scalable framework. In the process of creating this know-how we found a notable consequence: the code is pseudo-random, but still functional.
Random thoughts during a Cocktail Party
I grew up in the Italian-speaking part of Switzerland. It's a beautiful place, and an interesting mixture of Swiss quality and Italianità. You can imagine that a group of friends becomes chatty and loud after approximatively 2 seconds. The brain's ability of filtering out everything everyone else is saying, and pay attention only to one person is remarkable …and in this context it's essential!
How the brain solves this — the so-called "Cocktail Party" problem— remains largely a mystery. It is particularly challenging because it is not only a signal processing issue, but also cognitive: The brain needs to anticipate and reconstruct large portions of what is being said, which of course depends on the context of the conversation, as well as the setting, etc.
Up to a certain point, also neural communication suffers from similar effects. Neurons are very noisy structures, and there are tens of thousands of them in a cubic millimeter. Nonetheless, information is carried robustly, even for a long time. Neurons transmit information in many ways, and a unifying transmission code has not yet been discovered. Certain neurons modulate the number of spikes per second, others prefer waiting for the right moment.
Extracellular communication showed us that there isn’t a unifying coding, but a multitude of possible codes. More towards infinite possible codes, actually, because each pair of transmitter and receiver can use any code. The reason for this ability is that dendrites can easily learn extracellular sequences, any sequence. Numenta has recently shown a similar result, showing that each neuron can learn 100 or more patterns. Our current proof of concept can currently learn 10²⁸⁷ different sequences (running on an iPhone). Having "random thoughts" might not be so rare after all.
Because the code is free, the emitter and receiver can adapt to their specific noise environment. If the sequence is sufficiently long, noise becomes largely irrelevant. The wired connections, much stronger, act as directors and differentiators. [I left this part deliberately short; An in-depth discussion will follow]
Being largely independent of noise at the neural level doesn't explain the Cocktail Party effect, although it is clear that if a dendrite receives part of a sequence, with additional attention it might be able to figure out the rest by itself, just by looking at what's missing. Combining a series of such systems might lead to better listeners.
Addressing any part of the brain, without a map
Once the sequence has reached the dendrite, it can be processed in many ways. It can be combined with other signals to create action potentials, inhibit activity, or be carried on in some form. The latter is a very intriguing possibility.
If a sequence is relaid from one cell to the other for longer distances, it means that the final recipient gets a fingerprint of the original emitter. We have shown in a proof-of-concept that it is possible that the recipient can use this information as an address, and is then able to send information back to precisely that original emitter. Sending information back to a specific location can, of course, also be done with dedicated wired connections, but the difference is that the extracellular sequence method is dynamic and programmable: the extracellular sequence modifies the synaptic activation to reach a specific destination. This concept implies that the brain has a dynamic, floating addressing system, in which any group of neurons can address specific regions of the brain. Additionally, the same wired connection can carry different origin addresses. Talk about reusability of systems!
There are 3 interesting consequences to dynamic addressing:
- Facilitation of thought experiments
- Multi-modal sensory activation through diffusion
- Attention through anticipation of which region will be activated
(1) Dynamic addressing enables Selective Feedback Activation, that is eliciting or favoring activation in lower regions of the brain. It is well known that when you close your eyes and imagine a donut, your visual cortex activates to represent this donut —pink glazing with multicolor sprinkles on top [sorry for addressing your visual cortex so directly]— as if you were seeing it, just at a lower intensity. Providing higher regions of the brain — where more abstract information is processed — with "rolodexes" allows them to address specific regions without needing dedicated wired connections. The advantage is that the brain can build temporary connections, allowing it to simulate perceptions, scenarios, and receive a result out of it. In other words, Selective Feedback Activation is a mechanism for carrying out thought experiments.
Your short-term memory kept it there. But it didn't just keep the image, there was also a representation of its taste. And smell, and touch, and… Using dynamic addresses instead of dedicated wired connections allows to easily make experiments, even weird combinations that can't be learned from real-life experience, like a donut that tastes like spinach [ew!].
Zen meditation around a donut. All of this is subversive.
(2) Our real-life experiences are naturally multi-sensorial, and create archetypal representations: the donut is just sweet, and soft. One important aspect in the creation of these representations is their near-synchronicity: the shape, color, and softness is still vivid when its taste is perceived. Thus, when the neurons that are forming a representation of the donut are excited by all the relevant senses, their rolodex can retain a group of dynamic addresses together. In turn, this allows for, once a visualisation of the donut is being created in our minds, that the other senses will be automatically addressed as well. In other words, the dynamic addressing mechanism facilitates multi-sensorial representations, the feeling of what something is like, without specifically attending to all sensory areas.
(3) Attention indeed is per definition exclusive. Our brains are only capable of attending to 6–8 objects at the same time. The mechanism of how these objects are selected uses bottom-up saliency and top-down influence. Top-down influence remains largely a mystery, and dynamic addressing can offer a key of interpretation: as high-level neural structures know the addresses, they can provide diffuse extracellular facilitation by directing more resources towards specific regions. Compared to a connectionist explanation, this has the advantage of knowing at the high-level which areas will be targeted, therefore anticipating the needs, and possibly winning over and pooling other structures to focus even more.
The dense paradigm
In summary, we proceeded top-down, setting a hypothesis first, then validating through proof-of-concept, data, and value-added applications. However, the characteristics we have found were sufficiently interesting and valuable to go down the rabbit hole:
- The connectionists' view, including Turing's, is incomplete. Neural activity also propagates in the extracellular medium.
- Extracellular activity is more diffuse, like the signal from an antenna, and it can provide context by inhibiting or exciting the surrounding components. The activity can be directed through appropriate positioning of sources and sinks, and cannot be replicated in wiring.
- The spikes in neurons create waves of extracellular current that relay the message. The original sequence can be picked up by dendrites, at learned locations. The consequence of this mechanism is ad hoc coding: a specific code is created individually between emitter and receiver.
- A sufficiently long code can be passed on from on neuron to the next, providing higher-level brain regions with the information of the origin of each signal. This enables a dynamic addressing mechanism for carrying out thought experiments, multi-modal sensory activation through diffusion, and attention enhancement through anticipation.
A serendipity machine of sorts.
What does this mean?
First, there are waves of donuts traversing your brain. Right now.
Second, a mechanism for sending multiple coded messages through one wire increases dramatically the power of each region of the brain. Conversely, it imposes a limit on bandwidth (if the same wires and extracellular space are used for different representations, the number of learnable sequences depends on their length), a problem which doesn't exist if you have dedicated lines for each representation.
Third, it is a system of computational mechanisms, and therefore it can be explored directly, and the best parts retained for specific applications.
Fourth, the most important aspects propose a two-fold paradigm shift:
- Extracellular communication is as important as intercellular activity for information transmission, and
- The extracellular communication principle allows a group of neurons to address any other region of the brain directly.
This dense communication paradigm might be disproven and corrected by neurophysiology, and this will be welcome, as our intent was and is to explore alternate computational systems, and translate their benefits into further applications. The mechanism discussed here comply with the requirements to go beyond Machine Learning.