Mimetics is to Deep Learning as Memetics is to Deep Ingenuity

Carlos E. Perez
Jan 16 · 2 min read
Photo by Fatih Kılıç on Unsplash

Erik Hoel (@erikhoel) wrote a wonderful essay (The Semantic Apocalypse) about the astonishing ability of Deep Learning to mimic human creativity. Erik Hoel is an exceptional writer, who greatly exceeds my mediocre ability. I do highly recommend that everyone intrigued by Deep Learning to read his essay. Erik lends an uncanny insight into how Deep Learning will affect human society in a profound way. He writes about Deep Learning writing systems like GPT-2:

To call this a magic trick is too denigrating: this thing really can write poetry, and it really can use language, there’s just no intentionality or reference behind its language use.

In an earlier version, he had incorrectly used the word ‘memetic’ instead of a ‘mimetic’ to describe the cognitive ability of Deep Learning systems. It’s easy to make this mistake since the word ‘memetic’ is more pervasive in present-day internet culture. Erik likely originally intended to use Mimetic. Mimetic is, however, a real word in the dictionary and memetic is not.

The one vowel letter difference in these two words is huge. Mimetic describes the mirroring of syntax. Memetics describes the propagation and adoption of semantics. One could thus make an analogy that mimetics is to Deep Learning as memetics is to (some yet to develop) Deep Ingenuity.

Demis Hassabis of DeepMind partitions creativity into 3 forms. Interpolative, extrapolative and inventive. Interpolation is exhibited by Deep Learning in that it is able to recognize the statistical regularities of its training set. Extrapolative creativity is exhibited by Reinforcement Learning systems with self-play like AlphaZero. Self-play makes possible the automated creation of new training data. That is, new interactions discovered in play beget new ways to respond in play. Inventive creativity is still an open research topic, however, the creation of viral ‘memes’ is a reflection of this capability.

One last thought. There’s a lot of creativity in mimicry:

Further Reading

Carlos E. Perez

Written by

Author of Artificial Intuition and the Deep Learning Playbook — http://linkedin.com/in/ceperez https://twitter.com/IntuitMachine

Intuition Machine

Deep Learning Patterns, Methodology and Strategy

More From Medium

More from Intuition Machine

More from Intuition Machine

More from Intuition Machine

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade