Regression For Understanding Machine Learning
So why regression in machine learning. Regression help to understand how some of the machine learning algorithms work. Regression is a statistical process for estimating the relationships among variables. Such as how expenditure is dependent on income, According to the economic theory if income increases on an average expenditure also increase, now the question is how much. For finding the answer we need to develop a mathematical and statistical model.
Y = a+ bX + u
Here a, b and u is a random variable and some error term respectively Y is income and X is an expenditure. Say value of income and expenditure is given now if we can estimate the value of a, b and u we can precisely predict how much expenditure will change for changing income. There is some assumption such as Gauss-Markov theorem and other things which need to follow for regression analysis to work correctly but we ignore them for brevity. There is a method called ordinary least square (OLS) for estimating co-efficient ‘a’ and ‘b’ .
If we put the value of ‘b’ in the previous equation we can get the value of ‘a’. Now if we fit the value of a and b in our model we will essentially find a relation between estimated X any estimated Y. So we can predict on an average how much change Y for any change in X. But the question is how much the prediction will be accurate. If we the value of X and Y and draw a line which is most representative, we will get something like this.
But Accuracy of prediction will depend on the stochastic error terms. Our goal will be minimizing the stochastic error for improving the accuracy of prediction. For better accuracy, we will need tweaking many things.
Now here is the point of machine learning algorithm, the goal is minimizing error means minimizing the difference between actual value and calculated value using a various technique similar to regression analysis such as various kind of gradient descent.
My goal here is to show how machine algorithm works using linear regression and other various techniques.
