Deep Learning (Part 30)-Forward Propagation in a Deep Network
📚Chapter5:Deep Neural Network
Introduction
In the last tutorial, we described what is a deep L-layer neural network and also talked about the notation we use to describe such networks. In this tutorial, you see how you can perform forward propagation, in a deep network. As usual, let’s first go over what forward propagation will look like for a single training example x, and then later on we’ll talk about the vectorized version, where you want to carry out forward propagation on the entire training set at the same time.
Sections
Forward propagation in a deep network using Single Example
Forward propagation in a deep network using many Example
Summary
Section 1- Forward propagation in a deep network using Single Example
In the forward propagation algorithm, the input data is fed into the neural network through the input layer, which consists of neurons that represent the features of the data. Each neuron in the input layer is connected to neurons in the subsequent layers through weighted connections, which determine the strength of the signal being passed between neurons. As the input data passes through the network, it undergoes a series of transformations in the hidden layers, where the neurons apply activation functions to the weighted inputs to produce an output. The output of the forward propagation algorithm is the prediction made by the neural network based on the input data. This prediction is generated by the neurons in the output layer, which use the transformed input data from the hidden layers to make a final decision or classification. By iteratively adjusting the weights of the connections between neurons during the training process, the neural network improves its ability to accurately predict outcomes and learn complex patterns in the input data.
But given a single training example x, here’s how you compute the activations of the first layer. So for this first layer, you compute z1 equals w1 times x plus b1. So w1 and b1 are the parameters that affect the activations in layer one. This is layer one of the neural network,
and then you compute the activations for that layer to be equal to g of z1. The activation function g depends on what layer you’re at and maybe what index set as the activation function from layer one. So if you do that, you’ve now computed the activations for layer one.
How about layer two? Say that layer. Well, you would then compute z2 equals w2 a1 plus b2. Then, so the activation of layer two is the y matrix times the outputs of layer one. So, it’s that value, plus the bias vector for layer two. Then a2 equals the activation function applied to z2. Okay? So that’s it for layer two, and so on and so forth. Until you get to the upper layer, that’s layer four.
Where you would have that z4 is equal to the parameters for that layer times the activations from the previous layer, plus that bias vector.
Then similarly, a4 equals g of z4.So, that’s how you compute your estimated output, y hat.
So, just one thing to notice, x here is also equal to a0, because the input feature vector x is also the activations of layer zero. So we scratch out x. When I cross out x and put a0 here, then all of these equations basically look the same.
The general rule is that zl is equal to wl times a of l minus 1 plus bl. It’s one there. And then, the activations for that layer is the activation function applied to the values of z. So, that’s the general forward propagation equation. So, we’ve done all this for a single training example.
Section 1- Forward propagation in a deep network using many Example
How about for doing it in a vectorized way for the whole training set at the same time? The equations look quite similar as before. For the first layer, you would have capital Z1 equals w1 times capital X plus b1. Then, A1 equals g of Z1. Bear in mind that X is equal to A0. These are just the training examples stacked in different columns. You could take this, let me scratch out X, they can put A0 there.
Then for the next layer, looks similar, Z2 equals w2 A1 plus b2 and A2 equals g of Z2. We’re just taking these vectors z or a and so on, and stacking them up. This is z vector for the first training example, z vector for the second training example, and so on, down to the nth training example, stacking these and columns and calling this capital Z. Similarly, for capital A, just as capital X. All the training examples are column vectors stack left to right. In this process,
you end up with y hat which is equal to g of Z4, this is also equal to A4. That’s the predictions on all of your training examples stacked horizontally. So just to summarize on notation,
I’m going to modify this up here. A notation allows us to replace lowercase z and a with the uppercase counterparts, is that already looks like a capital Z. That gives you the vectorized version of forward propagation that you carry out on the entire training set at a time, where A0 is X.
Now, if you look at this implementation of vectorization, it looks like that there is going to be a For loop here. So therefore l equals 1–4. For L equals 1 through capital L. Then you have to compute the activations for layer one, then layer two, then for layer three, and then the layer four. So, seems that there is a For loop here. I know that when implementing neural networks, we usually want to get rid of explicit For loops. But this is one place where I don’t think there’s any way to implement this without an explicit For loop. So when implementing forward propagation, it is perfectly okay to have a For loop to compute the activations for layer one, then layer two, then layer three, then layer four. No one knows, and I don’t think there is any way to do this without a For loop that goes from one to capital L, from one through the total number of layers in the neural network. So, in this place, it’s perfectly okay to have an explicit For loop.
Summary
So, that’s it for the notation for deep neural networks, as well as how to do forward propagation in these networks. If the pieces we’ve seen so far looks a little bit familiar to you, that’s because what we’re seeing is taking a piece very similar to what you’ve seen in the neural network with a single hidden layer and just repeating that more times. Now, it turns out that we implement a deep neural network, one of the ways to increase your odds of having a bug-free implementation is to think very systematic and carefully about the matrix dimensions you’re working with. So, when I’m trying to debug my own code, I’ll often pull a piece of paper, and just think carefully through, so the dimensions of the matrix I’m working with. Let’s see how you could do that in the next Tutorial.
Please Follow and 👏 Clap for the story courses teach to see latest updates on this story
If you want to learn more about these topics: Python, Machine Learning Data Science, Statistic For Machine learning, Linear Algebra for Machine learning Computer Vision and Research
Then Login and Enroll in Coursesteach to get fantastic content in the data field.
Stay tuned for our upcoming articles because we reach end to end ,where we will explore specific topics related to Deep Learning in more detail!
Remember, learning is a continuous process. So keep learning and keep creating and Sharing with others!💻✌️
Note: if you are a Deep Learning export and have some good suggestions to improve this blog please share through comments and contribute.
For More update about Deep Learning Please follow and enroll:
Course: Neural Networks and Deep Learning
Do you want to get into data science and AI and need help figuring out how? I can offer you research supervision and long-term career mentoring.
Skype: themushtaq48, email: mushtaqmsit@gmail.com
Contribution: We would love your help in making coursesteach community even better! If you want to contribute in some courses , or if you have any suggestions for improvement in any coursesteach content, feel free to contact and follow.
Together, let’s make this the best AI learning Community! 🚀
