Artificial Neural Networks; All You Need To Know In Just 6 Minutes.
We all know what AI is, it is a branch of computer science designed to make machines smart. Well, the Artificial Neural Network is the foundation of AI. ANN’s are designed to simulate how we humans think and process information; ANN’s use artificial NEURONS to teach a machine to do tasks that require intelligence.
A Little Neuroscience 🧬
Our brains are made up of billions of neurons. Neurons are what allow us to receive input, and produce output (movement, emotional reactions, etc..). I could explain neurons for hours, but that’s not what we are here for. Instead, we will go into the really important stuff, the neuron’s structure.
Dendrites are what you see at the very beginning of a neuron, they are what receive input. Think of Dendrites like a power cable, it receives electricity and sends it to whatever it is powering (not the same as a cable).
The Axon is what you see in the middle of the neuron. The axon acts as a pathway through which the electrical signals travel to be received by other neurons. The Axon terminal or the tentacles at the bottom of the neuron is what sends the signal that produces and outputs. You may be wondering “but what does neuroscience have to do with ANN?” Well, it has everything to do with it. Neurons are ANNs, but organic. Now, we get into the juicy stuff. Now we talk about ANNs.
So, what are ANNs?
An Artificial Neural Network is a type of deep learning AI that replicates how we humans process information. All Artificial Neural Networks have three parts to them: Input, secret middle section that we will discuss later, and output. Here is a video that talks about ANN:
ANN input unit:
As shown in the diagram above, the input layer or the 4 circles at the very left of the picture receives input and directly passes it on to the hidden layers. Pretty basic, but in my opinion, it’s the most basic part of the ANN.
Activations:
Each circle, or each Neuron has a number in them, this number is called the activation. Ideally, the activation should be a number between one and zero. At the beginning the activation could be any number, however through the use of activation functions we can forcibly “squeeze” the activation to be a number between 1 and 0. Activations help the ANN to learn complex patterns, the activation is what decides what is to be fired to the next “Neuron” in the ANN.
Weights:
Weights are parameters that help transform input data in the ANN’s hidden layer. Weights decide how much of an influence the input has on the output, in short weights are just numbers between 0 and 1 that represent the strength of the connections between the units. Weights are super important in ANNs because they help you get the desired output. (change the weight X much you get E).
Here is another video that i found that explained ANNs well:
Activation functions:
Activation functions transform the summed weighted input (each input is multiplied by the weight associated with the synapse that is connecting it to the neuron) into the desired activation (ex: a number between 1 and 0) of that node or output. The most significant aspect of activation functions is that they can add non-linearity to neural networks. Not all patterns are “linear” in a sense, so activations help solve non-linear problems.
Types of activation functions:
- ReLU (rectified linear unit) is a type of activation function that solves the vanishing gradient problem, which allows models to learn faster and better.
- Hyperbolic tangent is a zero-centred function, which means that it is easy to model inputs with strongly negative, neutral, or positive values. I
- The sigmoid function normalizes the output of each neuron (makes it either 1 or 0), the sigmoid also allows clear predictions. However, this function creates the vanishing gradient problem.
- Leaky ReLU is known to prevent dying ReLU problems. It has a small positive slope in the negative area so it enables backpropagation even negative input values.
- Softmax is a type of activation function that can handle multiple classes (int, str, float). Softmax also normalizes the outputs for each class, making it a number between 1 and 0.
- Swish is a new type of activation function that performs better than ReLU. Studies show that swish performed approximately 0.6 to 0.9% better than ReLU.
- Parametric ReLU allows negative slopes to be learned. With parametric ReLU, it is possible to perform backpropagation.
This was all just a basic explanation, if you want to learn more about activation functions visit this website.
Biases:
Baises are extremely important when training an ANN, they help the ANN to learn efficiently and effectively. Biases are essentially constant numbers in the neural network that can shift the activation to the left or right. Think of biases as the value “b” in the equation Y=m(x)+b (linear line equation), it is a fixed number that effects the total “Y”.
The Hidden Layers (activation layers):
Essentially, activation layers perform nonlinear transformations by using the inputs entered in the neural network. Nonlinear transformations mean taking the weighted sums, and biases and putting them through an activation function. Activation layers are designed to produce an output for an intended result. For example, if you want your outputs to be squished into the numbers 1 or 0, simply change the activation function into a sigmoid function.
The Output Unit:
The Output for an ANN is the product of all the calculations made by the neural network. The output unit is pretty basic. All you have to know is that the output unit shows the output of the ANN; this completes my explanation of ANNs!
Companies that use ANN:
Applications of ANN and what the future might look like
ANN is a world-changing technology with the power of recognition (speech, image, etc..), and doing things that require intelligence (most of the things that require intelligence, like stock prediction or even complex calculations). To learn more about ANN applications visit this website.
I foresee a bright future for ANN. A lot of people suggest that AI will one day walk amongst us, will have feelings like us, and will maybe even surpass us. I believe that humanity will one day work together with AI to solve the world’s most pressing problems.
TL;DR
- ANNs are designed to mimic how me humans think and feel.
- ANNs are extremely similar to the neurons in our brains.
- ANNs are made up of 3 main parts: input, activation layer, and output.
- many popular companies like Google, Microsoft, IBM, and Intel use ANNs.
- ANNs is deep learning.
