9 Misconceptions About Deep Learning
We hear and read in the popular media about Artificial Intelligence (AI) all the time. We have movies about them. We hear about Elon Musk and Stephen Hawking warning us about AI’s apocalyptic consequences. We hear from the World Economics Forum about AI’s effect on taking away our jobs. We hear about how disruptive AI will be for businesses. However, when we listen to experts speak about this, they bring about an entirely different phrase: “Deep Learning”.
Deep Learning, two simple words that we all can understand, but yet in the context of Artificial Intelligence, when these words are combined they become inscrutable to the uninformed. In fact, even for the informed it is inscrutable. That’s because decades worth of statistical training have become a liability in understanding what it means.
Here are 10 points that you need to understand Deep Learning. It is simple, you just need to understand what it is not, and understand what it is.
- Deep Learning is not Good Old Fashion AI (GOFAI)
Expert systems, semantic web and deductive logic systems are examples of systems that are based on symbolic logic. These systems are typically associated with AI. They all do work, however they have one shortcoming: They are unable to effectively learn from the data.
2. Deep Learning is Radically Different from Machine Learning
Machine Learning in its most basic distillation is “curve fitting”. That is, if you have an algorithm that is able to find the best fit of your mathematical model with observed data, then that’s Machine Learning. DL at its earlier incarnation was about “curve fitting”, however it has progressed beyond that in recent years. Deep Learning Meta-Learning should be a big indicator to anyone that this is indeed very different.
3. Deep Learning does not mimic Biological Brains
The architecture of DL have are nowhere close to a biological neuron in structure. Even in behavior they are different. Biological neurons work on spiking behavior, DL system work in a continuous dynamical system. Some DL systems use Artificial Neural Networks, but that is just historic terminology that exists to this day. Anyone explaining DL in terms of biological neurons really doesn’t know what they are talking about. DL isn’t designed to ‘mimic’ biology, DL just happens to be a computational architecture that learns surprisingly well.
4. Deep Learning is not Artificial General Intelligence
DL can do some fantastic things like cross translate between different human languages and read out captions from images. However, the intelligence is really specialized and narrow. Sure DL can drive cars, but that’s nowhere near the capability of AGI.
5. Deep Learning is not “Just Math”
There was a Wired article titled “Deep Learning isn’t a Dangerous Genie, it is Just Math”. This is really the most vacuous statement I’ve heard! It is like saying that computers are just boolean circuits or brains are just made up of neurons or DL are made up of layers that are described using mathematical functions. It doesn’t explain the emergent complex behavior you find in computers, brains and DL systems.
6. Deep Learning is not Statistics
Classical statistics is about analyzing data using aggregate measures. DL systems however work in a domain that statistical methods do not apply. That is high-dimensional data with high mutual information among the variables. Simplifying i.i.d. (i.e. Independent and identically distributed) assumptions are simply not applicable.
7. Deep Learning is not Big Data
Big Data is a technology that is based on the idea that if you are able to store and compute through a massive amount of data, typically hosted in hundreds or thousands of off-the-shelf computers, then you can gain insight. DL is an algorithm that can sit on a single machine and can incrementally, special emphasis on incrementally, process your data to learn from it. Big Data can crunch massive amounts of data, but just because you can process a lot of data doesn’t mean you can derive insight or learn from the data. One last point, unlike Big Data, DL doesn’t need a lot of data to be useful.
8. Deep Learning is not understood by Data Scientists
Data Scientists are trained to do modeling of data, feature engineering and data analysis. DL just does what a Data Scientist does but without a human in the loop. This is actually a bit of an exaggeration. The reality is that most Data Scientists trained in other methods have not come up to speed with DL techniques.
9. Deep Learning is not just Artificial Neural Networks or Multi-Level Perceptrons
ANN or MLPs were developed way back in the 1950s. DL systems originate from this earlier work, however in recent years they’ve evolved to new kinds of models like Convolution networks, Long Short Term Memory, Residual Networks etc. The field has a much richer collection of concepts than existed when you studied it in graduate school.
10. Deep Learning is the reason for the current AI hype
Finally, this is where the greatest confusion exists. On a daily basis, the press continues to report the amazing progress of AI. Furthermore, you hear about firms like Google and Microsoft changing their entire software DNA to move into AI. The reason for this massive migration is because of Deep Learning. The big problem for the majority of the readers is that, the phrase itself “Deep Learning” is just too difficult to comprehend.
I hope this gives you a frame of reference, a lay of the land, a sketch of where exactly Deep Learning does not fit. You might still be perplexed about reading this considering I haven’t defined Deep Learning it all! My apologies, but unfortunately it’s a complex subject, however more detail can be found at Design Patterns for Deep Learning or start a conversation at FaceBook or LinkedIn.
However, if you are pressed for time and need one sentence to describe Deep Learning, it is just “Non-Equilibrium Information Dynamics”
