Brain inspired Computer Architecture
I know that I am intelligent, because I know that I know nothing
Ironically we know very less about our abler of knowledge.
Note : The aim of this blog is to make you aware of all the horizontals that exists without going in depth of any vertical!
Note Again : You may come across a lot of unknown terms, don’t get scared, open a respective Google tab and continue reading.
What we know …..
Brain, the organ that separates Simple and Rudimentary beings (me) from Complex & Intelligent beings (my maths professors, so they claimed). Brain is result of a large group of nerve cells (neurons) forming a large cluster. Acting in sync to perform varies functions enabling survival of organisms.
Nature isn’t Simple …..
But not all organisms (like Jelly-fish, Sea Star, Sea Cucumber etc) don’t have these clusters, hence we say they have no brain. While, technically true, these organisms still have presence of nerve cells (mostly all over their body with varying density of neurons).
That’s why we have a pseudo-second brain, the Enteric Nervous System (more on that later).
So it’s safe all complex organisms do need nervous activities to sustain life (though I don’t trust nature when it comes to surprises).
To Rule them all …..
The most interesting and intriguing are the brain of Human and Cephalopod. The latter is much less studied so we’ll focus on human brain.
Human brain is a marvel to say at least (yes even your sibling’s brain, I know you don’t trust me right now).
How Intriguing you ask …..
Have a look at these numbers:
- 10^14 : number of Synapses in our brain
- 10^109 : number of Seconds humans live
- 10^105 : bits of information processed per second (roughly 1 Peta-bit)
Human V/S Nature …..
Comparing efficiency of human brain to a computer is tough and tricky because human brain is designed to do General Purpose Computing (It can compute over a large variety of tasks which are very different in nature very efficiently).
Whereas Computer are generally designed to compute specific types, which they do very efficiently.
Yet to give a rough idea:
- K Computer : The Japanese supercomputer with computation capability of 10.5 petaflops took 40 mins to compute 1 Pb of data.
- Sunway TaihuLight : The current fastest supercomputer capable of 93 petaflops of computation power will take approx 4 mins to compute 1 Pb of data
- Human Brain : 1 sec …. enough said
The Zeus of Intelligence …..
Human brain is a titan force to recon with because of it’s sheer Volume.
The human brain consists of about 200 billion nerve cells (neurons) that are linked together by trillions of connections called synapses.
As the tiny electrical impulses shoot across each neuron, they have to travel through these synapses, each of which contains about 1000 different switches that route that electrical impulse.
In total, one human brain could contain hundreds of trillions of these neural pathways.
It can literally stretch from here to Jupiter ….. “SURPRISE SURPRISE %$#!@” said mother NATURE!
All THIS IS UTILISES 20W OF POWER ONLY !!
Human Creativity …..
Most widely used Architecture, currently, is Von Neumann architecture.
It consists of four principal components:
b) Input/output (I/O)
c) Arithmetic/logic unit (ALU)
d) Control unit (CU)
Vaguely, in Von Neumann architecture:
- Programs and data are held in memory
- Processor and memory are separate
- Data moves between the two but sequentially (opposite of parallel)
Limitations of Human Creativity …..
- Processor speeds have increased significantly
- Transfer speeds of Memory is significantly slower
- So the amount of idle time of processor has increased (as it waits for data to be fetched from memory)
- No matter how fast a given processor can work, in effect it is limited to the rate of transfer allowed by memory
In this configuration, latency is unavoidable. Hence a bottleneck is formed.
Even though there exist many alternate architectures and number of optimisations, this bottleneck still exists to some extent and creates a prominent bottleneck for the overall performance gains.
Creativity meets Nature…..
“Brain is the best motivation for Artificial Intelligence but not the Aim”
Agreed but awfully a lot of times we return to our brain’s architecture for optimisations.
There is essentially, at least, 6 order of magnitude (10⁶) difference between computational efficiency (computations per watt) of Brain and best Super Computers. GPU or TPU will barely reduce this gap to order of 5 magnitude.
Neuromorphic architecture is similar to structure in our brain. In theory, it can remove most bottlenecks. That, with current architectures, are unavoidable.
In last 2 Decades following architectures were developed:
Show Time …..
Neuromorphic Chips have pretty fascinating features:
- Sub-threshold Analog Logics (pico-amp) can be used to create extremely low power circuits able to emulate the non-linearity of brain (intelligent learning)
- Asynchronous circuit, don’t use clock signal, and the state of the circuit changes as soon as the inputs change. Since asynchronous circuits don’t have to wait for a clock pulse to begin processing inputs, they can be faster than synchronous circuits. Even completely Digital circuits.
- Memristor (a type of nanowire) is an electrical component that limits or regulates the flow of electrical current in a circuit and remembers the amount of charge that has previously flowed through it (like synapse).
Neurogrid, Show Time …..
(Well without going into ANY details. Like NONE!)
Each nanowire junction (orange dots) on this chip allows synaptic properties (learning by changing weights).
These chips use Address-Event representation.
These chips communicate when inputs (green and blue boxes) change a.k.a ‘event’.
They communicate the ‘address’ of the junction that fired to the neighbouring chips whose history is saved.
This Grid (crossbar) is capable of holding immense information compared to a traditional Neural Network architecture!
This grid simulates one million neurons and six billion synapses in real time!
This grid is also very power efficient!
More on this structure can be found in this paper.
Humans are incredible creatures. Their introspection and understanding of their own self, hold the key to almost preposterous potential.
Yes, again, even your siblings ( -_-)