What is Gradient/Slope? and How to Calculate One in Python (SymPy)

this article is a note to what i have learned so far (by the time i published). kindly suggest an edit if you spot any errors.

here i will talk about

• how to find gradient given 2 points
• how to find gradient of linear function
• how to find gradient of non linear function
• how to find gradient with 2 or more variables (partial derivative)
• how to interpret gradient (converting gradient to degree)

the concept of derivative is extremely important in machine learning. when building a predictive model, one will seek the best model with the smallest error. derivative helps machine learning models find that minimum error.

in mathematics, derivative is used to find the gradient of a curve or to measure steepness. it is also called the rate of change. so we take the change in y and divide that with the change in x. let’s just see what it looks in visual

first things first, let’s import the required libraries

import numpy as np
import matplotlib.pyplot as plt
import sympy as sy

gradient from 2 points

x = [1,3,5,7,9,11]
y = [2,4,6,8,10,12]
plt.plot(x,y)
plt.xlabel('X')
plt.ylabel('Y');

we are going to find the gradient of this line given two points

plt.plot(x,y)
plt.plot(x[2],y[2],label = 'point 1',marker = 'o')
plt.plot(x[4],y[4],label = 'point 2',marker = 'o')
plt.plot([x[2],x[4]],[y[2],y[2]],linewidth = 2,label = 'change in x (dx)')
plt.plot([x[4],x[4]],[y[2],y[4]],linewidth = 2,label = 'change in y (dy)')
plt.xlabel('X')
plt.ylabel('Y')
plt.legend();

point 1 = (5,6)
point 2 = (9,10)

change in y (dy) = 10 - 6
change in x (dx) = 9 - 5

gradient of a linear function

suppose the equation y=0.5x+3 as a road.

x = np.linspace(0,10,100)
y = 0.5*x+3
plt.plot(x,y)
plt.xlabel('length (km)')
plt.ylabel('height (m)');

we can use derivative to measure how steep the road is.

here we have y=0.5x+3 as the equation. we are going to find the derivative/gradient using sympy library.

#specify only the symbols in the equation
X = sy.symbols('x')
#find the gradient by using diff method
gradient = sy.diff(0.5*X+3)
print(gradient)
0.500000000000000

now we can see that the slope or the steepness of that linear equation is 0.5.

gradient of non linear function

let’s do another example with a non linear equation.

y = 0.5x²+3x-1

#generate 100 random numbers ranging from -30 to 20
x = np.linspace(-30,20,100) 
y = 0.5*x**2+3*x-1.

it’s a bit different from linear equation. each point in the curve produced by the equation will have different slopes. so we can only interpret the slope for a specific x value. in linear equation though, we can interpret the slope for the whole line. here we pick one point, if we want to determine the gradient or the slope.

plt.plot(x,y)
plt.plot(x[20],y[20],marker = 'o',markersize = 10, color = 'black');

suppose we want to know the slope of that black point. that black point holds -19.9 for the x value. in order to do that, we do the same procedure as in linear equation. take the function and use sympy to get the derivative.

X = sy.symbols('x')
func = 0.5*X**2+3*X-1
sy.diff(func)

1.0 x + 3

the output here is still in the form of a function. to find the slope of a certain x value, we just have to plug in the x value into the function.

suppose x = -19.9

slope = 1.0x+3
slope = 1.0(-19.9)+3

slope = 1.0*-19.9 + 3
print(slope)
-16.9

when x equals -19.9, the curve is descending at -16.9 rate.

gradient of 2 or more variables

for the example of this part, we will use the equation of squared residuals. it is one of the most popular error/cost function used in the field of regression.

we will find the 2 gradients of this function.

• the gradient with respect to m
• the gradient with respect to b

let’s first initialize all the symbols and create an instance for the function

x,y,m,b = sy.symbols('x y m b')
func = (y-(m*x+b))**2

now let’s find the gradient of the error function with respect to m

sy.diff(func,b)

plug in the variables, you will get the gradient of the error function with respect to b.

secondly, find the gradient of the error function with respect to m

sy.diff(func,m)

again, plug in the variables, you will get the gradient of the error function with respect to m.

how to interpret gradient

in the second part, we get 0.5 gradient. this means when you move forward by 1 meter, there will be a rise in height by 0.5 meter.

positive gradient means the line is going up or getting higher as you move forward. whereas negative means the line is descending or getting lower as you move forward.

we can also interpret the slope in a different way. we usually use angle to describe steepness, because it’s more intuitive. we can do that in numpy. all we need is the radian of the slope and convert it to degree.

0 degree angle is a flat horizontal surface, no steepness. you won’t have any problem walking on this one.

90 degree angle is a vertical surface, extreme steepness, very steep. you need to be a ninja from the leaf village to walk on surface like this.

now, let’s see how steep the slope of 0.5 is, in terms of the angle

first we use np.arctan(slope) to find the radian of the slope
second we use np.rad2deg(radian) to convert the radian to degree

radian = np.arctan(0.5)
angle = np.rad2deg(radian)
print(f'the slope of 0.5 has a {round(angle)} degree angle')

the slope of 0.5 has a 27 degree angle

conclusion

that’s all pretty much what i know about how to find the gradient of a function in python.

this derivative concept is used to find the gradient of a cost or error function of a machine learning model, to tell the model to which direction it should update the weights. and that direction is determined by the gradient of the error.

the best gradient is when the line produced by the error function has no steepness A.K.A zero gradient. thus an ML model will try to optimize so that the error gradient can get as close as it can to zero. in other terms, it cannot descend anymore (at its minimum). this procedure is broadly known as gradient descent.