Stochastic Gradient Descent
Ever wondered of a problem which involves huge data and you have to iterate that one by one and conclude to a conclusion. It will be really a very hectic process & nearly impossible.
To solve the category of above stated problems, we have a tool/algorithm stochastic gradient descent. In order to understand that, the very first step is to understand Gradient Descent algorithm in detail. The best article/blog for understanding gradient descent is:
After you have learned about gradient descent, now we can proceed in the direction of understanding stochastic gradient descent. Let’s proceed.
Explanation of Stochastic Gradient Descent
Consider that you are given a task of calculating the weight of each & every person living on this Earth. Will it be possible for you to do that task (obviously not!).
So, what a possible solution can be done, is that you can take average weight of let’s say 10000 persons, and can conclude that the calculated weight is the average weight for each person living on this Earth.
Same approach is implemented in Stochastic Gradient Descent, if there is very huge dataset, then Stochastic Gradient Descent will just take a random sample from that and calculate appropriate weights for them, and then those calculated weights are then used for rest of the data also.
This approach is very helpful as it reduces the consumption of the resources of the machine, and leads to quick result.
This algorithm is available as pre-built optimizer in the Keras Deep Learning library. It can be imported by the syntax given below to be used in deep learning:
“from keras.optimizers import SGD”
This is all for explanation of this Stochastic Gradient Descent. I hope this blog explains the algorithm in the best way possible.
