Python implementation of batch gradient descent

Recently I spent some times on learning Linear Regression and particularly implementing the gradient descent algorithm in Python. Some of you may already be aware of that there are some python libraries to support linear regression algorithm such as `sklearn`. But you won’t understand the logic behind the scene. I’d like to write the algorithm by `numpy` and `pandas` in order to fully understand this logic.

A little bit about Maths

The objective of linear regression is to minimise the cost function:

where the hypothesis is given by the linear model:

The parameter in this model is θ values. These are the values we need to minimise the cost function J(θ). One way to do this is to use batch gradient decent algorithm. In batch gradient decent, the values are updated during each iteration:

With each iteration, the parameter θ comes closer to the optimal values that will achieve the lowest cost J(θ).

The different about batch gradient decent algorithm is that it computes values for the whole dataset. This is great for convex, or relatively smooth error manifolds. In this case, we move somewhat directly towards an optimum solution, either local or global. Additionally, batch gradient descent, given an annealed learning rate…