The Deep Learning Roadmap

Carlos E. Perez
Apr 13, 2017 · 3 min read

It just occurred to me, that after a couple of years tracking Deep Learning developments, that nobody has even bothered to create a map of what’s going on! So I quickly decided to come up with a Deep Learning roadmap. A word of warning, this is just a partial map and doesn’t cover the latest developments. Many of the ideas I write on this blog isn’t even covered by this map. Anyway, here’s a start of this and hope people start coming out of their labs to further expand on it.

The “Unsupervised Learning” part is from a talk by Russ Salakhutdinov and the “Reinforcement Learning” part is from a talk by Pieter Abbeel.

There are a ton of other ideas that are coming out of the edges as well as the center of this diagram. Also, I did not show the connections between the 3 center concepts. For example, you can use CNNs for Value Iteration and GAN and VAEs use DL networks. It’s a wild world in the Deep Learning space and you just never know how all of this gets re-arranged.

I’ve got a higher level map that starts off with this:

that possibly can stitch everything together in one “grand unified theory”. This is how I think it will all play out:

Unsupervised learning is the the ‘dark matter’ where we need a lot more clarity. It’s my conjecture that meta-learning (with context) is the approach to this. There is some evidence that is developing, but I cannot know for sure. Modular Deep Learning is already in the cards. There is sufficient evidence that this works well. Market Driven Coordination is still early stages, but I believe that the only real way forward is to have diverse architectures working on the same problem and “markets” are a known decentralized way to coordinate actions.

There’s still a lot to be done though and we just in the early stages of Deep Learning evolution:

see: https://medium.com/intuitionmachine/five-levels-of-capability-of-deep-learning-ai-4ac1d4a9f2be

One additional key issue outside of unsupervised learning is the need to bridge the semantic gap between connectionist and symbolic architectures.

If you think there’s a demand for more clarity in Deep Learning, then support this kind of effort by buying the “Deep Learning Playbook”.

More coverage here: https://deeplearningplaybook.com

Other reading: https://www.ibm.com/watson/advantage-reports/future-of-artificial-intelligence/ai-innovation-equation.html

Intuition Machine

Deep Learning Patterns, Methodology and Strategy

Carlos E. Perez

Written by

Author of Artificial Intuition and the Deep Learning Playbook — http://linkedin.com/in/ceperez https://twitter.com/IntuitMachine

Intuition Machine

Deep Learning Patterns, Methodology and Strategy

More From Medium

More from Intuition Machine

More from Intuition Machine

More from Intuition Machine

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade