What is Gradient Descent?

Overview

This tutorial is on the basics of gradient descent. It is also a continuation of the Intro to Machine Learning post, “What is Machine Learning?”, which can be found here.

So what is gradient descent?

Gradient descent is a method of finding the optimal weights for a model. We use the gradient descent algorithm to find the best machine learning model, with the lowest error and highest accuracy. A common explanation of gradient descent is the idea of standing on an uneven baseball field, blindfolded, and you want to find the lowest point of the field. Naturally, you will use your feet to inch your way to the lowest point on the field. Looking for any downward slope. Conceptually, this is what we are doing to minimize our error and find our best performing machine learning model.

How does this relate to our y = mx + b equation in the first tutorial?

We can calculate derivatives, our error, and update our weights (a.k.a. m and b).

Example

Let’s get started. The only two libraries we will be using are numpy and matplotlib. Numpy is a great library for mathematical computations, whereas matplotlib is used for visualizations and graphing.

Importing libraries we will be using

# Numpy is a powerful library in Python to do mathemetical computations
import numpy as np…