Linear Regression-First step into fascinating world of Machine Learning
Artificial Intelligence has gained momentum in the recent years because of improvements of computing capacity. Under this wide spectrum of Artificial Intelligence there exists sub branch of Machine Learning which has gained lot of attention in the recent past. Machine Learning is nothing but training the machine according to our problem statement so that it can remember historical values and predict future values. This process can be simply called as training and testing of Machine Learning Models. Machine Learning is used in almost every industry to predict outcomes. For example, Machine Learning is used to identify if a tumour is malignant or not in medical field, it is also used by recruiters to check if a resume is suitable or not for a particular job profile. Machine Learning is also used to predict customer churn rate for business developments. For anyone who wishes to start studying Machine Learning Algorithms the first start point would definitely be Linear Regression Algorithm. Thus we can do the same by beginning our journey by understanding Linear Regression.
What is Linear Regression?
Before moving onto Linear Regression let us first understand what a Regression is. Regression is a method which is used to model dependent values based on independent values. Types of Regression techniques differs depending on the number and type of dependent and independent variables. This method is mostly used whenever there is a need to forecast and find out cause and effect between the variables.
Simple Linear Regression is used when there is only one X variable(predictor variable) to predict Y variable(target variable).It is used only when there exists linear relationship between predictor and target variables.The blue line in image represents best fit line.Best fit line is the optimal line which has minimum total error in predicting data points.Simple Linear Regression is usually denoted by:
Y=B_0+B_1*X
B_0 is the Intercept that decides where the line intercepts Y-axis
B_1 defines slope of the line
The ultimate motive of Linear Regression is to find the best values for B_0 and B_1.Let us explore the two most important concepts of Cost Function and Gradient Descendent before jumping into the algorithm itself.
Cost Function:
The cost function is used to help us find out the best possible values for our B_1(Slope) and B_0(Intercept) which in order helps us to arrive at our best fit line. Typically in this problem, we wish to find the best values in order to arrive at the best fit line which can be possible only when the total error is minimized. Thus this problem can be converted from a search problem to a minimization problem where the total error between predicted and actual values is the least.
We choose the above function to minimize total error in the prediction. We are squaring the error difference and sum over all data points and divide that total by the total count of data points which provides us mean squared error over all the data points. So this cost function is also called popularly as Mean Squared Error(MSE). Then this MSE function is used to modify values for slope and intercept such that MSE value settles at the least value.
Gradient Descent
Gradient Descent is one of the most required concepts to understand Linear Regression. It is the most common method of updating B_1 and B_0 to reduce the cost function. The main concept behind Gradient Descent is iteratively changing values of slope and intercept to reduce the cost function.
Assumptions of Linear Regression:
1) X and Y is assumed to have linear relation. If the data does not prove to show linear relation transformation techniques like log transformation is done to make it linear.
2) It assumes Input data is noiseless so it is necessary to remove outliers before training the model
3) Remove collinearity in order to prevent overfitting of highly correlated data
4) Make sure all the variables follow Gaussian distribution
5) Prediction is more reliable if you rescale input using scaling techniques
Conclusion
Linear regression is an algorithm every data science enthusiast must know to start building a base in Machine Learning. It is very simple but can be used in multiple scenarios. Hoping that this article was helpful to all of you out there!!!
