Erik Hoel (@erikhoel) wrote a wonderful essay (The Semantic Apocalypse) about the astonishing ability of Deep Learning to mimic human creativity. Erik Hoel is an exceptional writer, who greatly exceeds my mediocre ability. I do highly recommend that everyone intrigued by Deep Learning to read his essay. Erik lends an uncanny insight into how Deep Learning will affect human society in a profound way. He writes about Deep Learning writing systems like GPT-2:
To call this a magic trick is too denigrating: this thing really can write poetry, and it really can use language, there’s just no intentionality or reference behind its language use.
In an earlier version, he had incorrectly used the word ‘memetic’ instead of a ‘mimetic’ to describe the cognitive ability of Deep Learning systems. It’s easy to make this mistake since the word ‘memetic’ is more pervasive in present-day internet culture. Erik likely originally intended to use Mimetic. Mimetic is, however, a real word in the dictionary and memetic is not.
Definition of MIMETIC
imitative; relating to, characterized by, or exhibiting mimicry... See the full definition
The one vowel letter difference in these two words is huge. Mimetic describes the mirroring of syntax. Memetics describes the propagation and adoption of semantics. One could thus make an analogy that mimetics is to Deep Learning as memetics is to (some yet to develop) Deep Ingenuity.
Where is the Artificial Ingenuity in Deep Learning?
What is ingenuity and where does it originate from?
Demis Hassabis of DeepMind partitions creativity into 3 forms. Interpolative, extrapolative and inventive. Interpolation is exhibited by Deep Learning in that it is able to recognize the statistical regularities of its training set. Extrapolative creativity is exhibited by Reinforcement Learning systems with self-play like AlphaZero. Self-play makes possible the automated creation of new training data. That is, new interactions discovered in play beget new ways to respond in play. Inventive creativity is still an open research topic, however, the creation of viral ‘memes’ is a reflection of this capability.
One last thought. There’s a lot of creativity in mimicry: