Memory and The Machine

I would like to talk about memory , the act of encoding,saving and remembering in a mostly nuts and bolts kind of way, specifically how machines do it, how humans seem to do it and why it matters for A.I. and the future of the human species.


Preface:

Both humans and machines commit trillions of memory acts on any given day, in essence recreating bits and pieces of the past or perhaps dragging them along the time side of the time space fabric ( I like to think it has a luxurious velvety feel ), within those acts or close nearby lies our identity, our hopes, dreams ,credit card numbers, emails, selfies, bits, bytes and deep down there somewhere commingled with some other stuff I believe lies the secret to human consciousness and perhaps that of machines…

A prototypical act of Memory consists of quite a few things if you think about it, there has to be some event or thing to be remembered, real,imagined or even virtual( a percept ) , a mechanism to capture such thing or event ( an encoder ), another one to preserve it through time (a persister ?), and finally another one for recreating the percept or thing at a future date by recalling or remembering it, and of course someone or something to experience the act, ( an agent ).

NOTE: This is neither a strict or technical C.S or Neuroscience article, although I've reviewed a lot of research from both camps through the years (important sources at the bottom ) conveying and organizing some recent ideas in mostly plain English and verbose code while I nurse a cold is my main goal.

A simple memory test:

Let’s explore further the act of memory in both humans and computers with a simple memory test, ready ?

Human: Can you remember something you ate recently ?

Computer: I want you to remember which one of the following blocks the human clicks or presses (human please help the computer by running the demo now)

The answer to both questions should be a “piece of cake”, you can always retake the test, as for the computer, a very common and simple way to store values across programming languages is the array, we are using Javascript since it is fairly accessible and popular:

var set = []; // an Array 
function learn(event){
set.push(event.target.name);
dump.innerHTML = JSON.stringify(set);
}
// Explanation/Notes: 
// Every time you click a block, the function learn is called.
// The name of the block is then pushed into the array set...
// The contents of the HTML dump element are then populated with
// the contents of the set array.
//
// Unlike most humans not using mnemonic devices, the array can both
// store and instantly/perfectly recall a very large number of
// elements and their order.
// Unlike most computers,humans tend to remember some things for all // their lifespan...and can enjoy eating cake.

There are of course other programmatic ways of preserving this information, and we will gloss for now over the fact that behind these magical arrays a complicated process is taking place,notice that like the memory of the cake it will vanish into oblivion as soon as the demo is reloaded ,the computer is turned off or you get a more memorable slice or meal, so more importantly to our conversation, a memory has a certain permanence that lasts anywhere from seconds to years and lifetimes. Once you dissect an act of memory this way, you can’t help but notice they are everywhere, tree rings and half eaten onion rings, coffee stains and heartaches, books and carbon dating, and on and on,look around you and you will find those bits and pieces of the past everywhere.

I haven’t gone too far, but I hope at least by this point we are in the same page, we have a basic idea of what a memory looks like in both the computer world and the human one so we can now look more into the details.

Down the Memory Rabbit Hole.

Memory in humans can hardly be considered a unitary concept, by that I mean that we bunch together a lot of things that deviate from the prototypical act or have certain specific qualities into one single word, memories you see have intricate and active lives, are not all created equal and can be organized by subject, type,lifespan and maturity to name a few. In computers most of this knowledge has yet to be implemented, and it is one of the many things I like experimenting for fun, maybe even profit.

To illustrate this point, let me introduce you to a second type of memory with a new memory test, ready ?

Human: Out of these edibles which one is usually RED:

( bananas, chocolate,lettuce, cherry, pasta)

Computer: I want you to remember only the pair of blocks clicked by the human when they belong to a different layer,don’t remember single blocks or a pair of blocks from the same layer, human,please give the computer a hand by starting the demo and clicking randomly on the blocks.

And the relevant learning algorithm:

var set = [];
var assoc = [];
var layerA = [];
var layerB = [];
function learnAssoc(event) {
if (assoc.length === 0) {
// assoc is empty, so push element
assoc.push(event.target.name);
} else {
if (assoc.length <= 1) {
// assoc is no bigger than 2
if (assoc.indexOf(event.target.name) === -1) {
// assoc doesn't already contain element
if (layerA.indexOf(assoc[0]) >=0 &&
layerA.indexOf(event.target.name) === -1 ) {
//test assoc[0] does it belong to layer A
// is new element not part of layer A then
assoc.push(event.target.name);
} else {
//test assoc[0] does it belong to layer B
// is new element not part of layer B then
if(layerB.indexOf(assoc[0]) >=0 &&
layerB.indexOf(event.target.name) === -1){
assoc.push(event.target.name);
}
}
}
}
}
if (assoc.length === 2){
// Here we push our temporary array
set.push(assoc);
assoc = []; // and clear it.
dump.innerHTML = JSON.stringify(set);
// And we recall
  }
}
// Explanation/Notes:
//
A lot is happening on this one function, the bulk is devoted to // setting rules and restrictions as to what the association memory // algorithm should allow and pay attention to.
// In computers we have to painstakingly define these rules and
// restrictions as well as translate or create world variables.
// Notice we also now have a temporary store array where we hold
// and evaluate input associations before passing them on to a final // array, a recurrent theme in humans as well.

Cherry then. The memory of 2 or more related things is usually called an association memory, every act of memory in essence is an association memory since precepts are always related to other things like time, space,actions and agents or in other words,events and things just do not exist in a vacuum, so there is always something to relate them with and make associations; in humans, associational memory is more often related to the association of ideas, pattern recognition and specific objects or situations that bring up a certain emotion or action or further ideas.

In the computer world,simple and not so simple types of association memory are everywhere and perhaps you will recognize them by their use in sites and databases whose job is to remember everything associated with your user name. Arrays of arrays (like in our demo),a tree or graph structure (XML,JSON) , and cells,rows and tables in databases (Excel,SQL,text files,etc) are usually employed.

The basic ingredients of memory.

The next chapter in memory acts is currently being written in both camps. On the human side, an accumulation of research,experiments and models are giving us a better understanding of how things work, on the computer side, knowledge about how the brain encodes and the rules by which memories are generated have been in turn applied and modified for commercial and scientific use via the now popular neural networks, machine learning and object recognition.

One of my pursuits is that of figuring out how does the brain learn and remember at the cellular and systems level and how to transplant this knowledge to software agents and maybe even imbue them with some of our humanity, in order to talk more about this, and example is in order, here is a very simple group of virtual neurons for you to play with.

// Explanation/Notes:
// There are 4 Neurons in this model
// Neuron 1 is a pacemaker neuron,it will
// fire periodically no matter what.
// Neurons 2,3, and 4 are feed forward neurons, that is
// they will fire provided they have enough juice and their
// connected precursor neuron fires at the same time.
// Neuron 3 fires at a slower rate.
// Neuron 4 at a higher rate.
// Play with the speed, even though it is not properly timed
// real brain networks fire close to the highest rates, ( anything // above 24 fps, or Hz is hard to discern), the real brain usually // goes up to 300 Hz (or cycles per second).

There be neurons, networks and new things.

Even though this simple model is not alive,does not remember or encode anything, it can allow you to imagine a system where hundreds of these micro circuits interact with each other and the environment, it also gives a peek into harder questions like what makes an engram or memory ( the firing of a number of individual neurons or the information in the simulation at certain point in time in the computer), qualia and binding problems ( neuron 3 is a good candidate for a subjective conscious experience or the integration in time of various sensory modalities , these problem have consumed books of research alone).

I’ve talked a lot and said few things, memory and cognition in both humans and computers are big subjects and their technological applications have just begun , I’ll hopefully have more to say about the subject in the future . I’ll leave you with my belief that the zeitgeist is ripe for new advances beyond self driving cars and personal assistants, miracles in their own right, but perhaps low hanging fruit, and whether by my hand or others, profound changes (for the better) to our way of living and place in this planet could be just around the corner…

References & further reading:

  • Memory has traditionally been studied by either Psychology or Neuroscience, as such it is hard to pinpoint a definite text, The Oxford Handbook of Memory ( Edited by Endel Tulving and Fergus I. M. Craik) is a comprehensive if slightly outdated bible, Cognitive Neuroscience: The Biology of the Mind (by Michael Gazzaniga, Richard B. Ivry, George R. Mangun) has a very readable overview. Memory: From Mind to Molecules (by Larry Squire (Author), Eric Kandel (Author) ) and In Search of Memory: The Emergence of a New Science of Mind (Eric R. Kandel) are both great deep dives into the general subject.
  • Artificial Intelligence: A Modern Approach is a comprehensive introduction to A.I.
  • The issue of Qualia and Consciousness is explored in depth in The Quest for Consciousness: A Neurobiological Approach (by Christof Koch).
  • The binding problem is explored in Rhythms of the brain (by György Buzsáki) and Networks of the Brain (by Olaf Sporns), both books are great reads in their own right/domain.

About the Author :

Eugenio N. Leon (Keno) is a budding Mexican Designer,Web Developer, Artist and Inventor, born in Mexico City and currently living the accidental immigrant life in California, you can find bits and pieces of his work at www.k3no.com which he really needs to update but will tell you there always seems to be other more important things to do, like meaningful complex problems,he sure loves those, if you have one of those, he wants in, or not, your call.

Thanks !