Intuition, Innovation and the Limits of Deep Learning Generalization
How is intuition developed? How does this lead to innovation? What does this have to do with Deep Learning?
Intuition like consciousness is something that we are all aware of its existence but likely have not investigated in enough detail to have a grounded understanding of its nature. In fact, I would say that there’s more research on the nature of consciousness than research on intuition. I’ve written earlier about a few research groups that have explored consciousness with respect to an artificial general intelligence, however I don’t think has been equivalently the same effort with the study of intuition.
My specific interest is in the study of artificial intuition. I introduced this topic in an earlier blog post. In summary, artificial intuition, applicable equally to intuition, has attributes that differ from the more studied form of cognition (i.e. logical, rational). It is my conviction that Deep Learning systems are artificial intuition systems in contrast to GOFAI which is based on rational cognition. This judgment call is based on the commonality of attributes of intuition and Deep Learning.
In this post however, I explore deeper an understanding of intuition and reveal in there exists blindspots that we have in developing more capable deep learning systems.
Bruce Kasanoff has a very short article on Forbes, “Intuition Is The Highest Form Of Intelligence” which spurred me to explore this idea further. The study of intuition is not entirely new. I wrote earlier how Dual Process Theory introduces the idea that the mind works using two different kinds of cognitive systems. Kahneman, (see “Thinking, Fast and Slow”) employs Dual Process theory to explore how humans can fall into biased and incorrect thinking as a consequence of the interplay of intuition and rationality.
There are however other investigators have studied intuition in greater detail. One of the most popular books written about intuition is arguable Malcolm Gladwell’s book “Blink: The Power of Thinking Without Thinking”. Gladwell’s analogy of intuition is this idea that the mind “perfected the art of filtering the very few factors that matter from an overwhelming number of variables”. I would argue that this is an imprecise analogy. Inferences or predictions made through intuition are in many cases extremely difficult to explain. The mind does not do “filtering”, rather, it performs predictions in a massive parallel manner that appears to be a process of reduction, but is entirely different.
Gladwell popularized though the idea of intuition, enough to compel Gerd Gigerenzer, director and researcher at Max Planck Institute for Human Development, to write his book “Gut Feelings: The Intelligence of the Unconscious” explaining his research in much greater detail. Gigerenzer defines intuitions as having the following attributes:
“Appear quickly in consciousness”,
“Their underlying reasons we are not fully aware of”
“Are strong enough to act upon.”
The open question though is, how does the brain improve intuitive thinking? Theo Humphries explores this in “Considering intuition in the context of Design and Psychology”. Humphries challenges the ideas of Psychologists that have tagged the notion of intuition “to the ‘fringes’ of the field of psychology, within the realms of parapsychology, telepathy and premonition “. From a design perspective this is surprising to him, and he provides an enumeration of his surprise:
Intuition, as understood from a design perspective, appears to be so important as one interacts with “the designed world”. As an example, of a poor and good intuitive design:
Intuitive designs that are labeled as ‘intuitive’ is an indicator of a mark of praise and intuition is a foundational concern for usability design. The paper explores the disconnect between the design and psychology community:
for psychologists ‘action that is not planned or premeditated, answers that come without reasons, understandings that cannot clearly and quickly be put into words, are stigmatised as essentially second rate.” However, for designers, and artists (designers closely associated kin) spontaneous action and tacit intuition are valued as vitally important.
Designers however do not always work purely from their gut. Expert designers apparently have better intuition than novice designers. Experts have gained their knowledge through experience. In many complex fields, this kind of experience is captured in explicit form through the use of Design Patterns (alternatively “Pattern Languages”):
The elements of this language are entities called patterns. Each pattern describes a problem that occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. — Christopher Alexander
Brains are pattern recognition machines and with sufficient experience we begin to recognize not only more patterns, but increasing more complex patterns. This is what separates experts from novices. To appreciate “pattern recognition” Steve Blank, of Lean Startup fame, has a short video that expresses this best:
Bruce Kasanoff has an equivalent insight:
If all you do is sit in a chair and trust your intuition, you are not exercising much intelligence. But if you take a deep dive into a subject and study numerous possibilities, you are exercising intelligence when your gut instinct tells you what is — and isn’t — important.
The practice of Design Patterns is that it captures these patterns (i.e. tacit knowledge or intuition) in a form that in a way that is collectively curated and communicable to a much wider audience. Rather than have each individual learn from experience on their own, collective wisdom is captured through Design Patterns. In some sense, Design Patterns are “intuition that is industrialized”.
Now that we have these collection of patterns (codified or tacit), how then does this arrive at insight or innovation? What are the ways that we can process patterns? Technology Review reviews a recent research paper “Mathematical Model Reveals the Patterns of How Innovations Arises”:
In the cited paper, that innovation is enabled by “the adjacent possible”. That is those patterns that are one step away from existing learned patterns. So rather than developing patterns that have no connection, new patterns are realized through existing patterns and the thus new areas of unexplored patterns are discovered:
by providing the first quantitative characterization of the dynamics of correlated novelties, could be a starting point for a deeper understanding of the different nature of triggering events (timeliness, scales, spreading, individual vs. collective properties) along with the signatures of the adjacent possible at the individual and collective level, its structure and its restructuring under individual innovative events.
Unexplored patterns include either:
- Novelties — Patterns that are easily imagined and expected.
- Innovations — Patterns that are entirely unexpected and hard to imagine.
Surprisingly enough, the statistical occurrence of innovations shows striking regularities that can be researched in greater depth.
This is where we make the segue (or is it quantum leap?) into Deep Learning and that cumbersome concept of generalization. Deep Learning networks are pattern recognition machines. In fact, they are constructed in a self-similar manner where pattern recognition exists at different scales. A self-similar fractal structure of recognition machines. That is, Deep Learning systems consist of collections of pattern recognition machines that are also composed of collection of pattern recognition machines. A recursive structure that terminates at each neuron which themselves are recognition machines.
They are designed to recognized patterns based on previously learned patterns. Generalization, in a general sense (or should I have used “broad” instead?) is the recognition of new unseen patterns. The conjecture here however is that the class of unseen patterns are either of the class that is “easily imagined and expected” or even better “an entirely unexpected and hard to imagine” class. That is novelties versus innovations. The paper above seems to indicate that the mechanisms to discover the latter is the same mechanism as that of the former:
The same model accounts for both phenomenon. It seems that the pattern behind the way we discover novelties — new songs, books, etc. — is the same as the pattern behind the way innovations emerge from the adjacent possible.
This is thus something extremely intriguing, if we assume that novelty discovery an intrinsic capability (i.e. generalizability) of DL systems, then perhaps so is innovative discovery. That is, a capability that goes well beyond what I had expected!
To summarize, intuition is a cognitive mechanism that performs massive parallel pattern recognition to arrive at predictions. Generalization is an emergent behavior that arises through the combination of recognized adjacent patterns. These patterns may be of the novel kind (i.e. previously unseen) or the unexpected kind (i.e. unexpected). We conjecture that we can leverage the concept of the ‘adjacent possible’ as an abstract explanation for generalization.
One other curious characteristic of intuition is that is has a timelessness quality to it. What I mean is that, when we put our intuition to work, we don’t have an predictability as to when it reaches a good insight. It’s like some backend parallel thinking machine that goes off on its own. How often do we experience discovering new insights by just sleeping on a problem? Our intuition seems to work overtime when our consciousness is not awake. On the flip side, it also works extremely quickly in a manner that is not observable by the conscious mind. This indicates to me that the parallel nature of intuition leaves it unable to accurately make sense of time.
I leave you with the following quotation from an interestingly titled article “How Victory of Google’s Go AI is Stroking fear in South Korea”:
“AlphaGo actually does have an intuition,” Google co-founder Sergey Brin told New Scientist hours after his firm’s series-clinching third victory, which he’d flown in to witness. “It makes beautiful moves. It even creates more beautiful moves than most of us could think of.”
To get updates, subscribe to our newsletter https://www.getrevue.co/profile/intuitionmachine.
