PyTorch for Deep Learning — AutoGrad and Simple Linear Regression

PyTorch’s AutoGrad

PyTorch’s AutoGrad is a very powerful feature with which we can easily find the differentiation of a variable with respect to another. This comes handy while calculating gradients for gradient descent algorithm

How to use this feature

first and foremost , let’s import the necessary libraries

#importing the libraries
import torch
import numpy as np
import matplotlib.pyplot as plt
import random
x = torch.tensor(5.) #some data
w = torch.tensor(4.,requires_grad=True) #weight ( slope )
b = torch.tensor(2.,requires_grad=True) #bias (intercept)
y = x*w + b #equation of a line

y.backward() #letting pytorch know that Y is the variable that needs to be differentiated
print(w.grad,b.grad) #prints the derivative of Y with respect to w and b
output:
tensor(5.) tensor(1.)

This is the basic idea behind PyTorch’s AutoGrad.
the backward() function specify the variable to be differentiated
and the .grad prints the differentiation of that function with respect to the variable.

note: requires_grad parameter must be true for variables for which gradients are to be found

Simple Linear Regression With PyTorch AutoGrad

now that we have a basic understanding of the AutoGrad function, let’s use that to do linear regression with gradient descent for better understanding

I am creating a custom dataset on my own using some numpy functions

#creating a dataset
x = torch.tensor(np.arange(1,100,1))
y = (x*20+5+random.randint(-2,3)).reshape(-1)
print("shape of x: ",x.shape)
print("shape of y: ",y.shape)
output:
shape of x:  torch.Size([99]) 
shape of y:  torch.Size([99])

checkout the numpy documentation if you don’t know what a numpy function does

#initialising weight and bias term
w = torch.tensor(0.,requires_grad=True)
b = torch.tensor(0.,requires_grad=True)

specifying the number of iterations

epochs = 100

defining the forward loop

losses = []
for i in range(epochs):
  #making predictions
  y_pred = ((x*w)+b)
  y_pred.reshape(-1)
 
  #calculating loss
  loss = torch.square(y_pred - y).mean()
  losses.append(loss)
  #gradient descent
  loss.backward()
  with torch.no_grad():
    w  -= w.grad*0.0001
    b  -= b.grad*0.0001
w.grad.zero_()
b.grad.zero_()
#printing loss
if i%15==0:
  print(loss)
output:
tensor(1338702.5000, grad_fn=<MeanBackward0>) 
tensor(7.9803, grad_fn=<MeanBackward0>) 
tensor(7.9686, grad_fn=<MeanBackward0>) 
tensor(7.9568, grad_fn=<MeanBackward0>) 
tensor(7.9450, grad_fn=<MeanBackward0>) 
tensor(7.9333, grad_fn=<MeanBackward0>) 
tensor(7.9216, grad_fn=<MeanBackward0>)

Printing the final value of weight and bias

print(w.item(),b.item())
output:
20.115468978881836 0.34108221530914307

the value w of w is close to 20 and the value of b is somewhat close to 5.
so , we can conclude that our program learnt well using the PyTorch AutoGrad Function

Analyse the Loss

if you want, you could plot the loss over time using the losses list and see how the loss decreased.

Overview of what we did

1. We first created a dataset x and y. where we assigned the value of y to be equal to 20*x + 5
2. Initialised the weights and bias to 0 and set require_grad parameter to be true. (because , we are going to differentiate loss with respect to W and B to get the gradients )
3. define the forward loop where we make the predictions , calculate the loss, differentiate the loss with respect to w and b and get the gradients, update w and b and do this process continuously for 100 times
4. finally, print w and b to check if they are closer to 20 and 5, respectively (because our y = 20*x + 5) and those are the parameters we wanted our model to learn
5. The final value was really close, which proves our linear regression model worked well

Thank You