The Strange Loop in Deep Learning
Douglas Hofstadter in his book “I am a Strange Loop” coined this idea:
In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference.
— Douglas Hofstadter
Where he describes this self-referential mechanism as what describes the unique property of minds. The strange loop is a cyclic system that traverses several layers in a hierarchy. By moving through this cycle one finds oneself where one originally started.
Coincidentally enough, this ‘strange loop’ is in fact is the fundamental reason for what Yann LeCun describes as “the coolest idea in machine learning in the last twenty years.”
Loops are not typical in Deep Learning systems. These systems have conventionally been composed of acyclic graphs of computation layers. However, as we are all now beginning to discover, the employment of ‘feedback loops’ are creating one of the most mind-boggling new capabilities for automation. This is not hyperbole, this is happening today where researchers are training ‘narrow’ intelligence systems to create very capable specialist automation that surpass human capabilities.
Note: Original text migrated to the book. For more on this in this “strange loop” please consult:
♡ Please heart if you like this!