Mastering selective memorization

Photo credit: Yayoi Kusama

I often find myself having to re-learn old concepts in e.g. machine learning, mathematics and science as I explore new theories, models and methods, and expand my knowledge horizon. It’s not uncommon for me to realize that I need to refresh my memory, re-read formulas, and go again through material I once thought I knew well so that I can continue to make progress on a given topic.

To name a few examples, I recently found myself re-reading why the gradient can be understood as an expectation, how ABC works, the Bayesian interpretation of Lasso, or the role importance sampling methods play in Bayesian learning.

As I refresh concepts over time, I found it particularly helpful to actually memorize a few key formulas and concepts that help me continue learning effectively over time. While this may seem obvious, the “What-to-memorize” problem doesn’t seem to draw much attention from fields that rely heavily on hard facts (mathematicians, scientists, engineers). Perhaps I haven’t looked hard enough? Maybe.

Most interestingly, asymptotically I believe our learning efficiency is most optimal when we only need to resort to our implicit memory as we gather new information:

Implicit memory is one of the two main types of long-term human memory. It is acquired and used unconsciously, and can affect thoughts and behaviors.

Being aware of this problem has already helped me for example as I study deep reinforcement learning and principles of good management and leadership. When I encounter a new framework, theory or grand concept, I spend a few seconds thinking: what exactly should I remember from it? and I try to make a deliberate attempt to memorize that key nugget. I wish it was as simple as highlighting a few sentences, but I am optimistic we’ll get there some day :).

In short: Learning to identify what’s worth memorizing and mastering our implicit memory can make an important difference as we try to climb up what I call ladders of abstractions. In a sense, this is a learning to learn problem, which is a very active area of research in AI and ML (e.g. see here and here).

It’s worth noting that this is a hard problem even for people. Our brain surely does its best deciding whether to remember pointers or facts, and simultaneously deciding which ones to memorize. That said, I believe it pays off to be aware of these choices, andI am definitely looking for suggestions on how to improve them.