An overview of Deep Learning

Author : Parth Joshi , Btech Integrated , MPSTME , NMIMS

Under guidance of Seema Shah

References : https://en.wikipedia.org/wiki/Deep_learning , https://en.wikipedia.org/wiki/Artificial_neural_network , https://www.investopedia.com/terms/d/deep-learning.asp , https://www.quora.com/What-is-the-scope-of-deep-learning-in-the-future , https://en.wikipedia.org/wiki/File:AI-ML-DL.png

Deep learning (also known as deep structured learning or hierarchical learning) is an artificial intelligence function that imitates working of a human brain in processing data and creating patterns for use in decision making using methods based on artificial neural networks. Over time deep learning architectures lead to results which are comparable, or even superior in some cases, than the results produced by human experts in their respective architecture such as deep neural networks, computer vision, audio recognition, machine translation, bioinformatics etc.

Artificial Neural Networks (ANN) are capable of learning unsupervised from data that is unstructured or unlabeled. ANN are computing systems inspired by Biological Neural Networks that constitute animal brains. Such systems “learn” to perform tasks by considering examples, generally without being programmed with any task-specific rules. For example, in image recognition, they might learn to identify images that contain cats by analyzing example images that have been manually labeled as “cat” or “no cat” and using the results to identify cats in other images.

An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal from one artificial neuron to another. An artificial neuron that receives a signal can process it and then signal additional artificial neurons connected to it. The main objective of ANN was to solve problems in the way a human brain would, but over time its objective moved towards performing specific tasks. Deep Neural Networks (DNN) are a subpart of ANN which were used to further implementation of Deep Learning

An overview of the history of Deep Learning : The term Deep Learning was introduced to the machine learning community by Rina Dechter in 1986, and to artificial neural networks by Igor Aizenberg and colleagues in 2000, in the context of Boolean threshold neurons. The first general, working learning algorithm for supervised, deep, feedforward, multilayer perceptrons was published by Alexey Ivakhnenko and Lapa in 1965. A 1971 paper described a deep network with 8 layers trained by the group method of data handling algorithm. Later on many other deep learning architectures were introduced which made advances in dimensional viewing of objects from 2D objects to 3D objects. The impact of deep learning in industry began in the early 2000s, when CNNs already processed an estimated 10% to 20% of all the checks written in the US, according to Yann LeCun. Industrial applications of deep learning to large-scale speech recognition started around 2010. In 2010, researchers extended deep learning from TIMIT to large vocabulary speech recognition, by adopting large output layers of the DNN based on context-dependent HMM states constructed by decision trees. Advances in hardware enabled the renewed interest. In 2009, Nvidia was involved in what was called the “big bang” of deep learning, “as deep-learning neural networks were trained with Nvidia graphics processing units(GPUs).” That year, Google Brain used Nvidia GPUs to create capable DNNs. While there, Andrew Ng determined that GPUs could increase the speed of deep-learning systems by about 100 times. In particular, GPUs are well-suited for the matrix/vector math involved in machine learning. GPUs speed up training algorithms by orders of magnitude, reducing running times from weeks to days. Specialized hardware and algorithm optimizations can be used for efficient processing.

Deep Learning is one of the newest trends in Machine Learning and Artificial Intelligence research. It is also one of the most popular scientific research trends now-a-days. Deep learning methods have brought revolutionary advances in computer vision and machine learning. Every now and then, new and new deep learning techniques are being born, outperforming state-of-the-art machine learning and even existing deep learning techniques. In recent years, the world has seen many major breakthroughs in this field. Since deep learning is evolving at a huge speed, its kind of hard to keep track of the regular advances especially for new researchers. The future scope of Deep Learning is so much that the quantity cannot be expressed through words, there is a large amount of potential to be stumbled upon, research and experimented on to reach even higher technological advances. NVIDIA has increased the lifespan of Deep Learning by creating CUDA equivalent for Deep Neural Network (cuDNN). Thus Deep Learning has a scope to tackle wide variety of problem in near future.

Where deep learning lies in comparison to AI and ML