Artificial Neural Networks
We have seen lots of movies around AI where we are able to be fully engrossed into a world where AI becomes a friend, a lover, a killer, a soldier, a slave, and discover the power of its ability to achieve amazing things like creating bio-engineered replicas of humans and creating artificial intelligence for the betterment of military. Artificial Intelligence is truly the greatest invention of humanity. As seen in the movies, AI will either be the death of us, or allowing us to do incredible things.
AI stands for Artificial Intelligence. Intelligence means the ability to acquire and apply knowledge and skills and Artificial means made or produced by human beings.
“AI¹ : any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. A more elaborate definition characterizes AI as “a system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation.”
History of AI
To know about the history of AI. Let’s go back to the era between 1940–1960 and we will be able to see a handful of scientists from a variety of fields (mathematics, psychology, engineering, economics and political science) begin to discuss the possibility of creating an artificial brain and that’s when the journey started. People started to work on machine learning, Robotics, Neural Networks. In this blog we are going to focus on Neural Networks and see how it works internally.
So what is Artificial Neural Networks and how does it works?
Artificial Neural networks are inspired from the neural network in our brain. Scientists studied how our brain works and tried to simulate the way human brain analyzes and processes information.
Human brain contains about 100 billion basic units called neurons. Neurons are information messengers. They use electrical impulses and chemical signals to transmit information between different areas of the brain. Each neuron is connected to about 10⁴ other neurons. Synapses are the contact points where one neuron communicates with another. With this structure of neurons human brain can do many things like analyze, interpret data and can learn new things.
Similarly in artificial neural networks, artificial neuron is a mathematical function which takes one or more input data, processes it and transmit the signal to other neurons like synapses in human brain. With the help of these artificial neural networks we are trying to help machines do incredible things that human brain can do.
There are multiple applications of the artificial neural network’s let’s zoom into one of the application and let’s see how it works in detail.
Take example of identifying handwritten digits
For us it’s quite easy to identify handwritten digits. Different people write digits differently still we are able to identify digits. But if we are asked to write a program to identify digit’s where input will be say 28 * 28 grid then it will be relatively difficult task.
With the use of Neural Network’s Let’s see how we can solve the problem of identifying handwritten digits when provided as 28 * 28 grid.
Each element in the grid is called a Neuron. So we have a total 784 Neurons data which will be fed into the system.
Each Neuron holds the Gray Scale value of the corresponding pixel ranging from 0 for black pixel up to 1 for white pixel. This Gray Scale number is called Activation. So these 784 Neurons form the first layer for our Network.
Last Layer may contain 10 Neurons between 0 to 9 and again each neuron will have some Activation(some number between 0–1). So the brightest neuron with highest Activation in the last layer is what network thinks the given digit is.
This Network may have some Hidden Layers in between.
Activation in One Layer determines the activation in another layer. Imagine activations in one layer corresponds to some activation pattern in hidden layers which corresponds to the activations in the output layer and based on activations in the last layer you get your output.
Now why do we need all those layers? What are those hidden layers actually doing?
Take the example of recognizing digit 9. For recognition we should divide digits in smaller parts to understand it better. Like 9 has a loop at the top and a line below. Now 8 also has a loop at top but this time another loop at the bottom also.
In the first layer say all the neurons having loops at the top get activated and in the second layer 8 is eliminated as we need a line in the below part and this is how 9 will be activated in the output layer.
We can also introduce more layers in between to again break down the circle and straight line and make the network more accurate.
While doing so we will also assign some weights between our neuron and neuron in the first layer. Now based on the activations in the previous layer we will calculate their weighted sum according to their weights. So to identify any pattern, say loop, we will adjust our weights so that loop pixels are activated and all other surrounding pixels are not activated (i.e. towards 0).
We need our activation to be in the range of 0 and 1. To do this we need to use some function which will take our weighted sum and fit it into the scale between 0 to 1. This function is known as Sigmoid. You may also need your neuron to be activated not just when the weighted sum is greater than 0 but say when it’s greater than 10. To achieve this we can simply subtract that
number from our weighted sum. This number is called Bias. The bias value allows the activation function to be shifted to the left or right. Hence changes to the weights alter the steepness of the sigmoid curve, while bias shifts the entire curve so it fits better the data.
We have weights for each connection and bias for each neuron which gives us a lot of room to play with the network. We can tweak all the weights and biases to make the network behave differently and more accurately. Isn’t it awesome….
It’s not the end of learning once output is generated. AI Model will receive either a reward or penalty for how it produces the output and this model can even learn from it’s past experiences and can tweak itself to maximize the rewards. This is knows as reinforcement learning. This maybe the next topic for my blog :)
In this blog I have just covered a single use case for artificial neural networks. It can do many things other than simply identifying the handwritten digits.
Now a lot of frameworks and utilities are available to work with Neural Networks like Deeplearning4j², TensorFlow³, Keras. Download one of them and play with it.
Happy Coding Happy Learning…….
References
- AI : https://en.wikipedia.org/wiki/Artificial_intelligence
- Deeplearning4j (Open-source, distributed, deep learning library for the JVM): https://deeplearning4j.org/
- TensorFlow : https://www.tensorflow.org/tutorials
