Apr 22 · 5 min read

# Example:

Suppose that:

We have the following linear system

Where:

(𝑦̂) is predicted output

𝑠𝑙𝑜𝑝𝑒 = 𝛽1

𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 = 𝛽0

Where:

(𝑦) is actual output

Tip To get the idea of “Gradient Descent” we will work first on “intercept-coeff.” then on both of coefficients together (intercept & slope). So a fixed value for ‘slope’ will be used ( 𝛽1= 0.64)

## Calculations:

3. Compute the new-𝛽0.

4. Repeat steps from “1” to “3” with the value of (new-𝛽0) again and again till the absolute value of step to be equal to or less than value of 0.001 (|step| ≤ 0.001) i.e. at this value the change in both step and new-𝛽0 will be too small.

5. At (|step| ≤ 0.001), gradient-descent approaches its goal to get the best value of system’s intercept-coeff. (𝛽0) with the minimum error (cost).

And hence our target is approached.

The following code is doing this:

`import pandas as pdimport numpy as npimport matplotlib.pyplot as plt%matplotlib inline# Importing our dataset from CSV file:dataset = pd.read_csv('./Datasets/weights.csv')x = dataset['weight']y = dataset['height']# get relation between "intercept" coef. with squared error:def sqr_err():    sqError = []    incpt = np.arange(0, 2, 0.1)    slope = 0.64    for i in incpt:        y_pred = i + slope * x        residual = sum((y_pred - y)**2)        sqError.append(residual)    plt.plot(incpt, sqError)    plt.title('Intercept vs Squared_Error')    plt.xlabel('Intercept')    plt.ylabel('Squared Error')`

The above part represents the cost curve relationship with 𝛽0.

`# Draw Squared_Residual / Intercept curve:sqr_err()# Initial value of intercepti = 0learningrate = 0.1while(True):    # Draw a certain value of Squared_Error to get slope at    # any point on cost-curve    y_pred = i + 0.64 * x    cost = sum((y_pred - y)**2)    plt.plot(i, cost, c='r', marker='o')        print(f'Intercept= {round(i,4)}')    print(f'Seqared-Residual: {round(cost, 4)}')    # Draw tangent    costSlope = -2*(1.4 - (i + 0.64 * 0.5)) - 2*(1.9 - (i + 0.64 * 2.3)) - 2*(3.2 - (i + 0.64 * 2.9))    print(f'Cost_Slope: {round(costSlope, 4)}')    line_x_values = np.arange(0, 1.2, 0.1)        # tangent will pass through (0, tangent_intrcept) and (i, cost) and have slope "costSlope"    tangent_intrcept = cost - (i * costSlope)    tangent = tangent_intrcept + costSlope * line_x_values    plt.plot(line_x_values, tangent)        stepSize = learningrate * costSlope     if abs(stepSize) <= 0.001:        print(f'Absolute value of Step Size is less than (0.001): {round(abs(stepSize), 4)}')        break        print(f'Step Size: {round(stepSize, 4)}')    print('\n')# New Intercept = Old Intercept - Step Size    i = i - stepSize    plt.show()`

The Second part of this tutorial will be about how to use gradient descent to get both of linear system coefficients (𝛽0 & 𝛽1) of equation-1.1 in the same approach our target which is min. cost.

Part-2:

Written by

Written by