Dawn of Neural Networks — Explainable AI Visualization (Part 5)
This article continues the background overview of the research ‘Explainable Deep Learning and Visual Interpretability.’
In the mid-1990, Artificial neural networks were introduced by researchers, and they marked a paradigm shift of predictive modeling from the world of applied statistics toward computer science and machine learning.
Deep Learning Revolution
Deep learning is a superset of machine learning that has its root in mathematics, computer science and neuroscience. Above all, mathematics has an essential role in deep learning as it is in the study of visual processing of the brain. Neural networks were reincarnated around 2010 with “deep learning” as a new name, largely due to much faster and larger computing systems and some new ideas.
Deep learning is a specific set of techniques from the broader field of machine learning that focuses on studying and using deep artificial neural networks to learn structured data representations. It is used for classifying patterns using large training data sets and multi-layer neural networks.
Deep learning is primarily a method for machines to learn from data loosely modeled on how a biological brain learns to solve problems, where each artificial neural unit is connected to many other units. The links can be strengthened or weakened based on the data used to train the system. Each successive layer in a multi-layer network uses the output from the previous layer as input.
The origin of deep learning goes back to the dawn of artificial intelligence in the 1950s when there were two contrasting visions for how to create an AI: one vision was focused on symbolic approaches based on logic and computer programming, which dominated AI for decades; the other was based on learning directly from data, which took much longer to develop and show results.
In 1956, John McCarthy, a Mathematics Professor at Dartmouth College, proposed a workshop called Dartmouth Summer Research Project on Artificial Intelligence, which gave birth to the field of AI and motivated a generation of scientists and experts to explore the untapped potentials for information technology to match the capabilities of humans.
In 1962, Psychologist Frank Rosenblatt at Cornell University aimed to create brain analogs useful for analytical tasks. He invented a simple technique for simulating neurons in hardware and software. This marked the inception of the research in enabling machines to learn and classify.
Rosenblatt proposed ‘perceptron,’ a single-layer neural network for binary classification that could learn how to sort simple images into categories — for instance, squares and triangles. Perceptron became the basis of further research that culminated in creating multi-layer learning networks, which have formed the basis of modern deep learning.
In the last decade, deep learning has been successfully applied to various domains and applications that require large volumes of digital data for training and providing helpful information. Recently, they have been advancing the state-of-the-art in artificial intelligence and have led to crucial breakthroughs in many areas, such as computer vision (CV), speech recognition, and natural language processing.
Architecture Overview
Now that we have seen various components of deep neural networks, this section gives a broad overview of the four significant architectures of deep networks.
I survey neural network types to provide general insight and discuss some of the influential vision-based architecture of deep neural networks with applications in industry and academia.
Perceptrons
A neural network is a highly parameterized model inspired by the architecture of the human brain. It was widely promoted as a universal approximator — a machine with enough data to learn any smooth predictive relationship.
Single Layer Perceptrons
A single-layer perceptron or feed-forward neural network is a collection of neurons arranged in a sequence of sev layers. Each neuron receives input from the previous layer and performs a simple calculation (e.g., compute a weighted sum of the input followed by a nonlinear activation function). The network neurons collectively perform a nonlinear mapping from input to output data. This mapping feature is learned from the data by adjusting and adapting the weights of each neuron using a technique called backpropagation.
rives transformations of the inputs — nonlinear transformations of linear combinations — which are then used to model the output
The figure above shows a simple example of a feed-forward neural network diagram. There are four predictors or inputs, five hidden units and a single output unit:
The hidden layer derives transformations of the inputs — nonlinear transformations of linear combinations — which are then used to model the output, and there is a single output unit:
The next article in this series covers the structure and depth of the multi-layer perceptron and the convolutional neural network model used in computer vision:
https://medium.com/deepviz/explainable-ai-and-visual-interpretability-background-part-6-6467736f82b8
