Basics of Deep Learning

When I was young, I really like Lego. It was amazing for me because I can build anything with small blocks. I could build dragons, castles, and trains. There is no kid here but a grown man instead. I felt almost the same emotion from deep learning. There are basic blocks in deep learning and you can build anything you want. You can create autonomous driving, pictures, and drug candidates. I will explain the basic blocks and their glue to stick together in this post.

Forward & Backpropagation

We need to know how the neural net calculates the output or its error. It is really easy. You put the input and input layer toss the result of the calculation to the next hidden layer. The calculation consists of a linear function and a non-linear function, activation function. Each neuron represents one linear regression but it has an activation function at the end. If you don’t have activation functions, then it is just big linear regressions. We propagate the result from layer to layer until it reaches the output layer. At the output layer, we calculate the loss function to evaluate the loss. We have many parameters in each neuron and we need to figure out how much each neuron contributes the loss. Linear regression uses the Gradient Descent Method to calculate the loss. It is the same. We will use Gradient Descent but the specialty is from chain rule of derivation. If…