Linear Regression

“An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem.” — John Tukey

linear regression is a linear approach for modeling the relationship between a scalar dependent variable y and one or more or independent variables denoted X. Unlike Logistic Regression the dependent variable here can take a continues range of values. The relationships are modeled using linear predictor functions called Linear Hypothesis, whose unknown model parameters are estimated from the data.
Lets start with a linear hypothesis.

Where

We can write the same in concise notation as below.

The above equation signifies that given a new input feature vector x the predicted value of the output will be given by the hypothesis h. So the accuracy of the predictions depends on the weights of the input features.

If we denote actual output values for the m training examples by a vector y of size m. One reasonable way is to use training set to make value of h as close to y. We define the actual difference as cost function J given by.

So now the equation becomes find weights for which cost function J is minimum.

Let

Where
Y : is output vector for m training examples.
X : is matrix of size m*n where each ith row belongs to ith training set.
Q : is weight vector of size n for n training features.
So

And

Which results In

Example:
Given Training set here
X = number of claims
Y = total payment for all the claims in thousands of Swedish Kronor

Plot of Y w.r.t. X.

The above will Resolve to

And Finally the model will be

The plot of data along with model fitting the data.