Member-only story
Understanding Gradient Descent for Machine Learning
A deep dive into Batch, Stochastic, and Mini-Batch Gradient Descent algorithms using Python
Gradient descent is a popular optimization algorithm that is used in machine learning and deep learning models such as linear regression, logistic regression, and neural networks. It uses first-order derivatives iteratively to minimize the cost function by updating model coefficients (for regression) and weights (for neural networks).
In this article, we will delve into the mathematical theory of gradient descent and explore how to perform calculations using Python. We will examine various implementations including Batch Gradient Descent, Stochastic Gradient Descent, and Mini-Batch Gradient Descent, and assess their effectiveness on a range of test cases.
While following the article, you can check out the Jupyter Notebook on my GitHub for complete analysis and code.
Before a deep dive into gradient descent, let’s first go through the loss function.
What is Loss Function?
Loss or cost are used interchangeably to describe the error in a prediction. A loss value indicates how different a prediction is from the actual value and the loss function aggregates all the loss…
