How an Algorithm Was Made to Think Like the Brain

A subset of Artificial Intelligence, Machine Learning is an extremely broad field that solves numerous problems such as finding various groupings from data and using regression techniques for predictive analytics. This article focuses on a sub-category of Machine Learning called Deep Learning.

With discoveries made every day, Deep Learning is a fascinating field of research. Deep Learning is the method of using Artificial Neural Networks (ANNs) to learn from given data. A Neural Network is a type of algorithm inspired by the biological function of neurons in the brain. Here is an example of the most basic form of an Artificial Neural Network:

The Node

Each one of the red, yellow, and blue circles represents a node. The node is to a neural network what a neuron is to the brain. To understand this relation at a deep level, here is the biological neuron:

At a high level, similar to how a biological neuron cell receives information through its dendrites and passes that information to other neurons out of its axons through small gaps between neurons called a synapse, a neuron in an Artificial Neural Network is a computational unit that receives an input and outputs the resulting information through different connections to other neurons. A simple Artificial Neural Network architecture is as follows:

This vastly simplified neural network model takes a vector of inputs, performs computations with the vector of weights, denoted 𝜃, and passes the remaining value to an activation function. The activation function maps the inputted value to another value between 0 and 1. (More on this here)

How do they work?

When we hear about the amazing work done by neural networks, we always hear about these models “learning”. How exactly do neural networks learn? Well, hold your horses because we first have to discuss how inputs make their way to the output layer of a neural network.

The vastly simplified neural network in the paragraph above gave a massive hint as to how neural networks function. The input layer has many nodes, each representing an input value. Each connection from the input layer to the subsequent hidden layer, and from the subsequent hidden layers to the next hidden layers, has a weight. A weight is a numerical value that corresponds to how major of a change a particular input value will have on the rest of the model.

If the weight of the first connection from X₁ to the next layer is 0.75, the weight of the second connection from X₂ to the next layer is 0.15, and the weight of the third connection from X₃ to the next layer is 0.35, then we can see that any change in first connection will have the highest impact on the model. How exactly is this change applied? The computation from nodes of a previous layer to a node in the next layer is actually less complicated than one might assume.

Each value is multiplied by its corresponding weight and the sum value of this computation for each node is then added to a bias. We will call this value z₁(X). At this point z₁(X), will be passed through an activation function. (Since this article focuses on explaining how neural networks work, the actual activation function we select now does not matter. However, this might change for projects as different activation functions suit different tasks better.) The value output from the activation function will be written as a₁(X), where X = z₁(x) — essentially a₁(X) = a₁(z₁(X)). This logic is used for every node in a neural network, for each node in each layer. For example, if we were to have a second hidden layer with a single node, its computation would look something like a₂(z₂(a₁(z₁(X)) + β₂)). This process continues until all the nodes have been reached and a value is subsequently outputted.

Now that we know how neural networks return an output value for each input value, we can finally discuss how neural networks “learn”. Think back to quizzes in school, how does a student learn? The student will first complete the quiz and receive a score for their performance. For the questions that the student got wrong, they either refer to their course material or their teacher to find out why they were wrong.

This analogy applies well to neural networks. First, the neural network needs to know the different values of error for the output that is provided. Second, the neural network must update its weights to match the correct outputs from the data set.

Finding Error In Output

The first part of the “learning” phase of neural networks will be done through cost functions. In other words, a cost function takes the output of the neural network as input, then applies the value of the input to the function resulting in the output of the cost function as a numerical value describing how “off” the neural network model was. The most popular cost function is the Mean Squared Error (MSE) cost function.

Tuning the Model

To tune the model, some changes need to be made in the model so that it does not replicate the same mistakes. This tuning of the model happens through a process called Backpropagation using Gradient Descent. The model slowly analyzes the results from the MSE cost function, finds which weights corresponded to the lowest errors, and then makes changes to the current weights so that they are closer to the weights which had the lowest error.

Applications

Now that we have de-mystified the incredible neural networks, where can this be applied? Well, any task which does not require reasoning is a task for Neural Networks.

Such tasks range from detecting maladies in mere pictures of x-rays to detecting digits and words through handwritten digits, and even sentiment analysis from different social media posts. (In fact, I created a neural network to detect digits from handwritten images, you can check that out here)

More about me — my name is Rohan, I am a 16-year-old high school student learning about disruptive technologies and I’ve chosen to start with A.I. To reach me, contact me through my email or my LinkedIn. I’d be more than glad to provide any insight or to learn the insights that you may have. Additionally, I would appreciate it if you could join my monthly newsletter. Until the next article 👋!