Reality Check! Is Deep Learning really worth all the hype?
Deep Learning has, for at least the last 5 years, been at the very top of the list of buzzwords in technology. It has become so widely popular that the terms Artificial Intelligence and Deep Learning have become synonymous these days. A huge amount of hype has gone into what deep learning is and what it can do. But, as it often happens, most of these hypes are not exactly true. In this post, let us take a step back and look at what exactly deep learning does and what it can do.
What exactly is deep learning? (Really!)
Deep learning or neural network architectures have been used to solve a multitude of problems in various different fields like vision, natural language processing. Architectures like the LeNet, VGG-16, Inception have become part of the day to day toolkits of almost every practitioner out there. Some of there architectures have a very deep structure with a lot of complexity inbuilt in them. Take to look at the diagram of the Inception V3 network to convince yourself that it is indeed complex!
That’s all scary, but let us take a moment here to consider what a deep learning model really does. If you ask me, it does exactly what any other machine learning algorithm does. Basically, it converts a higher dimensional vector and converts it into a lower dimensional vector. The high dimensional vectors are the vectorized representations of the input data and the low dimensional vector is the output vector. In mathematical terms, a DeepNet is just a function which converts an input X to output Y. That’s it! It really is that simple, people!
Now, the natural question that arises is, what about that loopy structure before? What does that do exactly? Well, the answer to that is also simple. One of the main differences between machine learning and deep learning is that in machine learning, the feature extraction is done manually while in deep learning, the features are extracted by the model itself. In the structure of Inception, VGG, ResNet, etc the complex parts of the network are usually busy doing just feature extraction. After this, conceptually, a neural network does the same mapping as any other ML algorithm, plus maybe adds a little bit of non-linearity to it.
Deep learning is not enough!
Deep learning can in no way mimic human intelligence. We are still far from creating systems which have human-level intelligence. And I am of the opinion that deep learning alone will not be enough to achieve this feat. The main reasons for why I am of this opinion is listed below:
- Deep learning models don’t “understand” the data: A deep learning model does not “understand” data in the same way that we do. I would say, what a DL model does, is “memorizing” data. Any model in machine learning just sees numbers and spits out numbers. Real intelligence will only be achieved when the model is able to associate some “knowledge” with data. A model should “learn” from its environment and become better in time.
- Deep learning models require an insane amount of data: Almost everyone reading this will most probably know the amount of data it takes to train a deep model. For even a trivial task of building an image classifier to identify cats from dogs, the training stage takes about 2 GB of images, which is just too much for such a trivial task. We did not have to see 1000 dogs and cats to recognize which is which.
3. Deep learning models don’t generalize enough: Don’t get me wrong here. I agree that deep learning models are able to generalize reasonably. My argument here is that it is not enough. For true “intelligence”, we will need extreme generalization as compared to local generalization achieved by state of the art deep modes today.
Okay! So now what?
By this time, I hope I have convinced the readers why I think deep learning is not enough to achieve true “intelligence”. That still begs the question: so how do we achieve “intelligence”. Honestly, I don’t know for sure, but I have some ideas which may help increase the potency of learning algorithms.
- Finding a universal representation of data: One of the crazy ideas which I believe may help in achieving human-level intelligence is unifying the representation of data. If the agent saw and understood data as abstract objects with reasoning links connecting them, we will be able to make the agents learn these reasoning links and to have a rich “knowledge” of the world. Instead of concentrating on the “model”, maybe we should be concentrating on how to “represent” data to our model. If you think about it, the human mind creates such reasoning patterns and abstract models of the world around us, and that is a key reason for our “intelligence”
- Neural Communication: This is another idea that is inspired by the human brain. Our brain has separate, distinct regions for dealing with vision, speech, etc. Another key aspect of how our brain works is through communication between the various centers. If we can tie 2 or more neural net architectures in a way that they communicate the “knowledge” they obtain and collaborate amongst themselves, then we are a taking a step towards achieving human-AI.
- Lifelong Learning: The next generation of intelligent systems should be equipped with continuous learning. In terms of deep learning, think of a neural net which keeps on learning. It could successfully learn new links between objects, and most importantly, it will reduce the dependence of “intelligence” on abundant data and put it more into “evolving data”.
Moral of the story
To summarize, deep learning has too many limitations to actually mimic strong human-level AI. Deep learning has to be supplemented with concepts of abstract models, communication between models about these abstract constructs and a life long learning policy to be comparable to human-level intelligence. It is highly encouraging to know that researchers across the world have already started moving in these directions. Some of them, have already shown very convincing results. I hope this post has left you with some ideas which you can take home and ponder about. Till next time, adios.
