Linear Regression Part I
Introduction:
There are 3 types present in Machine Learning: Supervised, Unsupervised and Reinforcement learning. Linear Regression can be categorized under Supervised learning.
On knowing the historical data of the independent variable we can find the value of ‘y’ (dependent variable) which is called Supervised learning.
Sample: Let Independent variable be “x ” & Dependent variable be “y” x1 = Age, x2=Income & x3=Age of Building . We need to predict y, which is the price of the house. This process is carried out in Supervised learning.
Now coming to our topic Linear means “straight line”, Regression means “relationship between 2 points”. So Linear Regression can be stated as the straight line trying to predict the relationship between two points. This can be carried out in two ways: 1. Using Sum of Least Squares & 2. Using Gradient Descent technique. In this article we are going to see in detail about linear regression using Sum of Least Squares method.
Before getting into the algorithm, we need to know what is slope, intercept and equation for a straight line.
Slope can be stated as steepness of the line, mathematically it is change in Y with respect to X. If change in X = 2 & change in Y =1, then slope is defined as change in Y / change in X which is 2/1=2
Intercept is any point in line which meets the Y axis.
Equation of a straight line is y= mx + c, where “m” is the slope and “c” is the intercept, “x” is the independent variable & “y” is the dependent variable or the model.
Goal of simple Linear Regression is to find the “best fit line” . The line which is formed from the best estimates of y values with every given value of x.
Linear regression with Sum of Squares is carried out in 2 ways:
- One Variable &
- Two Variable technique
Linear Regression using One Variable:
Problem: We need to predict the height of the next person in queue
In this above dataset we have only 1 variable to operate with, which is “height”. Now we need to predict the next persons height, withought knowing their weight / other factor we cannot predict the value. Hence we need to replace the value with the mean of the above data.
Mean is 153, so the next predicted value/height of the next person will be 153.
Now we need to learn about another metric called SSE[Sum of Squared Errors]
How far the actual data points are differing from the mean[Actual value — Predicted value].We need to compute the difference, square them and sum it up.
Thereby always it is advisable to go with the mean, to predict the future value if we are dealing with one variable.
Now lets learn about Linear Regression with Two Variables:
Problem: We need to predict the height of the next person.
Now predicted value will be 1.346 times increased from the previous value. The intercept 87.876 does not makes any sense in the real world so that can be omitted for now.
Thereby from our sample data the minimum SSE is 47.17. The “best fit line” for predicting our future value is Y¡^= 1.346x + 87.876
Conclusion: As it was one and two variables in dataset we predicted the future values using excel easily. In real world the dataset might be having more than 1000 rows and columns, in that case it will be tough for us to find the line where error is minimum. An optimization algorithm called Gradient Descent is used usually, to make the process easier for huge dataset.
With this we have come to an end of this article! :)
Happy Learning! 🙂
Please wait for my next post which is on Linear Regression Part II, Gradient Descent approach
