Intuitive Learning using the Feynman Technique
There is a learning method that is attributed to Richard Feynman (aka “The Great Explainer”) coined the Feynman technique:
- Pick and study topic
- Pretend to teach your topic to a student
- Go back to the literature when you get stuck
- Simplify and use analogies
I don’t think that Feynman had explicitly described a “Feynman Technique”, but there are some hints that he had a preferences to many aspects of this learning approach. Biographer James Gleick in his book “Genius: The Life and Science of Richard Feynman.” Gleick writes:
“[He] opened a fresh notebook. On the title page he wrote: NOTEBOOK OF THINGS I DON’T KNOW ABOUT. For the first but not last time he reorganized his knowledge. He worked for weeks at disassembling each branch of physics, oiling the parts, and putting them back together, looking all the while for the raw edges and inconsistencies. He tried to find the essential kernels of each subject.”
Which describe a deconstruction and then construction approach to understanding a topic. This method of course is not unusual. Feynman himself ( an excerpt of the book published in a 1996 issue of Caltech’s Engineering & Science magazine ) remarked though about the need to explain complex ideas using simpler concepts:
Feynman was a truly great teacher. He prided himself on being able to devise ways to explain even the most profound ideas to beginning students. Once, I said to him, “Dick, explain to me, so that I can understand it, why spin one-half particles obey Fermi-Dirac statistics.” Sizing up his audience perfectly, Feynman said, “I’ll prepare a freshman lecture on it.” But he came back a few days later to say, “I couldn’t do it. I couldn’t reduce it to the freshman level. That means we don’t really understand it.”
Finally, written on his blackboard when he passed away in 1988:
It reads “What I cannot create, I do not understand.” Again emphasizing the requirement of generating an explanation from likely first principles.
To be subsequently quoted by OpenAI researchers to describe motivation of their Deep Learning generative model approach. The point though that they are making is that there seems to be a connection with understanding and the ability of re-generating a concept or idea. Although, GANs can generate realistic images, I highly doubt that GANs can understand any of it!
However, let’s go back to the method, since it indeed is illuminating in how the human mind works in learning as well as potentially how a Deep Learning system (a machine) may also improve its learning.
There are three key elements of the method that is worthing focusing on:
- The activity of explaining some complex topic. (Generate)
- The disassembly of a complex topic into simple terms. (Analogies perhaps?) (Decompose)
- The reconstruction of the topic looking seeking any inconsistencies. (Compose and Validate)
- The iterative nature of the method. (Iterate and Refine)
I have this thesis that humans and deep leaning machines are both intuition machines. What I mean by this is that evolution has bequeathed humans with a cognitive machine that reasons through induction and analogy. Our evolutionary machinery does not have specialized symbolic and deductive reasoning capabilities. Humans simply have not had sufficient evolutionary time to have evolved this capability. Rather, our intuition machinery is forever compensating in an inefficient manner to perform rational thought.
Unlike computers that can natively understand symbolic forms and can perform deductive reasoning at blinding speeds, our brains don’t have this specialized capabilities. This is one reason for the failure of the GOFAI approach to AI, where there was an assumption that humans thought can be built up from formal reasoning.
However, having evolved to be social beings. That is, humans are equipped with machinery that enables us to effectively function within social groups. Our brains allow us to (1) understand the behavior of members of our social group and (2) communicate and share our thoughts with members of our social group. Over time, humans have developed language that has persisted and evolved through many human generations. Our brains have learned to understand and communicate in the languages that we have been taught.
Language is basically information compression. Actually, the ability to generalize can be framed in terms also of information compression. Our ability to express our thoughts through language can also be a measure of our intelligence.
To recap, our brains have different kinds of intelligence (see: Re-inventing the cognitive stack):
The higher levels of this stack (i.e. logical, interpersonal, verbal and intra-personal) are supported by more primitive cognitive capabilities (i.e visual-spatial, rhythmic, sensory and motion). Our brains have built only approximations of these higher level cognitive capabilities. We pretend that we are indeed rational, but it actually takes us a lot of mental energy to work our way through a rational thought process. Our natural tendencies is to employ our intuition to make fast and sometimes error prone (or biased) judgements.
The point I want to make here is that, to learn something well, humans and intuition machines have to learn it at a basic visual-spatial, rhythmic, sensory level. Knowledge just doesn’t stick when conveyed as a symbolic and logical level. Our brains fundamentally can’t understand abstractions. Rather, our brains have metaphors of abstractions that are captured by very primitive constructs. According to George Lakoff:
You can only have meaningful thought through connections to the body.
If you have ever listened to one of Feynman’s many lectures, you will find how he takes a great effort of using analogies to explain complex topics. He engages our intuition to its fullest. He made popular the use of Feynman diagrams, a visual notation that represented complex particle physics interactions ( see tensor networks).
In addition, his approach to explaining dynamics employed the use of the idea of “path integrals”, also a highly intuitive representation of dynamics. Something that in fact can also be expressed visually:
Simply explained the dynamics of a system can expressed as the aggregation of the alternative paths from point A to point B. Clearly Feynman understood the value of engaging our intuition in the study of advanced abstract concepts.
The Feynman technique takes advantage of the cognitive tools that we humans are good at. That is, exercising our ability to understand simple conceptual ideas coupled with our learned ability to explain things through language. That is why, it is very important in this method to engage the communication ability of the brain. You actively change your perspective by trying to explain it to someone else (even if it is pretend). This engages the mind to the think in a certain way.
The act of explaining something is a natural way of model building for the mind. It places oneself from the perspective of another person. When we explain something we simultaneously attempt to understand the ideas while also sensing if the other person understand what we say. The engagement is a more intense mental activity than say, simply highlighting sections of text in a book. Our minds are simply chaotic systems where our consciousness takes great pains to manage. We can’t learn thing by just reading, we have to involve ourselves in more active engagement by the process of actually re-creating what we studied. There is no learning without effort and there is no effort without engagement.
However, we can very easily fall into the trap of the “Cargo Cult.” Another idea that Richard Feynman came up with. This is why “First Principle” thinking is extremely important. The above technique helps you understand complex topics, however it does not mean that your understanding is correct! Feynman said:
The first principle is that you must not fool yourself — and you are the easiest person to fool.
That is, one needs to examine the concepts that are used in one’s explanation and determine if you can break down these concepts to irreducible concepts. Furthermore, if one can verify the validity of each irreducible concept (usually applicable only in a few domains!). The validation part, is why Elon Musk remarked:
“I think its important to reason from first principles rather than by analogy”
It is easy to think in analogy. However, in many advanced technology fields, many concepts are counterintuitive. We simply cannot assume that reality operates in the same way that our primitive minds are used to. Thinking by analogy serves us very well in surviving in the jungles, but can be a problem with complex subjects.
The first principle, or rather “Feynman’s First Principle” is what gives grounding to our intuitive thinking. Intuition allows us to explore multiple alternative paths simultaneously to arrive at new thought patterns. However, it may contain errors and thus re-validation of our thoughts through higher rigor and rational thinking is a must.
The Indian mathematician Ramanujan, had an immense mathematical intuition that gave him the uncanny ability find mathematical generalizations of infinite series without rigorously deriving the details. Ramanujan’s intuition was unparalleled, he had no formal advance mathematics training and he was self-taught. Ramanujan had a exceptional intuition with numbers, however there were times where his intuition could only “see” so far. That is, certain infinite series will will work out for the first hundred instances, but eventually break down. Ramanujan had developed a mind with an unimaginably advanced intuition with respect to abstract algebra. However, advanced, that intuition had its limitations. Those limits could be verified more more rigorous rational thought (i.e. rigorous hand calculation).
This is the nature of intuition, it can be creative and fast, but at the same time fallible. The most gifted “computing machines” of our species are still unable to perform computations like computers do, rather they’ve developed their own intuition to perform unexplainable acts of mental gymnastics.
Neil Lawrence argues that the difference between human and machine intelligence comes down to embodiment factors. That is the ratio of the ability compute power over the communication bandwidth. Humans are not blessed with telepathy or the ability to mind-meld. Rather, we are restricted to language to communicate with other. In contrast, machines can interface through high-bandwidth channels and have massive computational capabilities. The embodiment factor that Lawrence ascribes to human is 10¹⁶ and for machines it is a mere 10.
So as a measure of progress of human intelligence of a machine, a machine needs to be trained with the constraints of a high embodiment factor. That is, a machine needs to be able to explain its thought processes. Alternative way of saying this is, a machine must be able to perform sufficient generalization that it can explain its conclusion in a constrained sequential language.
Update: A recent article by Wired shows a Physics professor teaches his students by having them create videos of physics problems they solve.
At first I was tempted to wait until I understood the idea to make a video, but this was a mistake. The videos aren’t just for assessment, they are for learning.”
