Linear Regression without iteration (Mathematical Intuition)

Let’s understand the problem statement before diving into the actual concept

Each regression problem is given X and Y values, and after training with a particular algorithm, it predicts the Y(i) based on the X(i). For example, let’s assume X = [1,2,3,4] and Y = [1,2,3,4], Thus, Y is the same as X. If X(i) equals 5, then Y(i) equals 5.

To establish a generalized formula for the above example, let’s consider the line equation as Y = mX + c ( m: slope, c: Y-intercept), as we already concluded that Y is the same as the X thus equation becomes Y = X (Where m = 1 and c = 0).

Let us now look at the mathematical intuition of linear regression. Let’s consider the same example as before, where X = [1,2,3,4] and Y = [1,2,3,4]. Here we need to derive Y = X (m = 1 and c = 0), so the main concern here is to find out the the m and c values.

To derive the “m” and “c” values, we will make use of the cost function. i.e,

Here, n = number of entries, and y = mx + c, Then we take the partial derivative of the cost function w.r.t. “m” and “c” and equate them to zero such that it results in two equations and two unknowns. Further, this can easily be solved to find the m and c values.

To find m,

To find c,

As a result of substituting m = 1 and c = 0 in the line equation, we get y = 1(x) + 0, Y = X.//