Uni-variate Linear Regression

Uni-variate linear regression focuses on determining relationship between one independent (explanatory variable) variable and one dependent variable.

In this we will implement prediction of a student scoring in an exam on the behalf of how much time did it spend in studying. We will optimise our parameters using gradient descent method, and calculate the loss with Mean Squared Error.

Here we will use student_score.csv as our Datasets.

First lets import all the necessary python libraries….

As you can see our data-frame contains 2 columns in which Hours in feature and Scores are the target. And we want to predict scores on the basis of hours.

Lets store out features and target value in x, y variables respectively.

The most important part of data analytics is analyse the data and visualise the data. Data visualisation is the graphic representation of data. Let’s see how our data looks..

Now Uni-variate linear regression predict on the basis of Equation of the line i.e y = mx + b

From this equation we have X and we need to calculate Y, but what we don’t have is the m and b i.e gradient or slope and intercept or coefficient. These are knows as Parameters of the ML model.

we can calculate the m and b from the Gradient Descent method. Gradient descent is an optimisation algorithm used to minimize cost function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we use gradient descent to update the parameters of our model. Parameters refer to coefficients in Linear Regression and weights in neural networks.

The gradient can be calculated as:

Lets see how..

Gradient Descent method subtracts a small portion of the m and b from m and b itself i.e derivatives of m(dm) and b(db). So every time a small position is going to be subtracted from the parameters so that these parameters lies in the steepest slop.

The size of these steps is called the learning rate(lr). It could be 0.1, 0.01, 0.001..etc depends upon your problem and no. of iteration you have to do.

With a high learning rate we can cover more ground each step, but we risk overshooting the lowest point since the slope of the hill is constantly changing. With a very low learning rate, we can confidently move in the direction of the negative gradient since we are recalculating it so frequently. A low learning rate is more precise, but calculating the gradient is time-consuming, so it will take us a very long time to get to the bottom.

Now we got our m and b, let’s store it in the variable..

Let’s predict..

What we have obtained is the prediction on the basis of the features or inputs we got.

A Loss Functions tells us “how good” our model is at making predictions for a given set of parameters. The cost function has its own curve and its own gradients. The slope of this curve tells us how to update our parameters to make the model more accurate.

The most common method to calculate loss is Mean Squared Error

Our sklearn library has pre in-built function to calculate Mean Squared Error. Let’s calculate the loss of our model.

The lesser is the loss the more accurate our model is. So here our loss is pretty low.