A First Shot at Deep Learning with PyTorch

Create a hello world for deep learning using PyTorch

Published in

DAIR.AI

4 min readDec 22, 2019

In this notebook, we are going to take a baby step into the world of deep learning using PyTorch. There are already a ton of notebooks out there that teach you about deep learning and PyTorch. My goal here is to provide a foundation and introduction to deep learning using PyTorch. Therefore, this notebook is targeting beginners but it can also serve as a review for more experienced developers.

After completion of this notebook, you are expected to know the basic components of training a basic neural network with PyTorch. I have also left a couple of exercises towards the end of the tutorial so as to encourage the learner to do more research and practice their deep learning skills.

Importing the libraries

Like with any other programming exercise, the first step is to import the necessary libraries. As we are going to be using Google Colab to program our neural network, we need to install and import the necessary PyTorch libraries. You can find the completed notebook at the end of this tutorial.

The Neural Network

Before building and training a neural network, the first step is to process and prepare the data. In this notebook, we are going to use synthetic data (i.e., fake data) so we won’t be using any real world data this time.

For the sake of simplicity, we are going to use the following input and output pairs converted to tensors, which is how data is typically represented in the world of deep learning. The x values represent the input of dimension (6,1) and the y values represent the output of similar dimension. The example is taken from this tutorial.

The objective of the neural network model that we are going to build and train is to automatically learn patterns that better characterise the relationship between the x and y values. Essentially, the model learns the relationship that exists between inputs and outputs which can then be used to predict the corresponding y value for any given input x.

The Neural Network Components

As said earlier, we are going to first define and build out the components of our neural network before training the model.

Model

Typically, when building a neural network model, we define the layers and weights which form the basic components of the model. Below we show an example of how to define a layer named layer1 with size (1, 1). For the purpose of this tutorial, we won't explicitly define the weights and allow the built-in functions provided by PyTorch to handle that part for us. By the way, the nn.Linear(...) function applies a linear transformation to the data that was provided as its input. We ignore the bias for now by setting bias=False.

Loss and Optimizer

The loss function, nn.MSELoss(), is in charge of letting the model know how good it has learned the relationship between the input and output. The optimizer (in this case an SGD) primary role is to minimize or lower that loss value as it tunes its weights via training.

Training the Neural Network Model

We have all the components we need to train our model. Below is the code used to train our model.

In simple terms, we train the model by feeding it the input and output pairs for a couple of rounds (i.e., epoch). After a series of forward and backward steps, the model somewhat learns the relationship between x and y values. This is notable by the decrease in the computed loss. For a more detailed explanation of this code check out this tutorial.

Testing the Model

After training the model, we have the ability to test the model predictive capability by passing it an input. Below is a simple example of how you could achieve this with our model. The result we obtained aligns with the results obtained in this notebook, which inspired this entire tutorial.

The output is: 17.096769332885742

Final Words

Congratulations! In this tutorial you learned how to train a simple neural network using PyTorch. You also learned about the basic components that make up a neural network model such as the hidden layer, optimizer, and loss function. We then trained the model and tested its predictive capabilities. You are well on your way to become more knowledgeable about deep learning and PyTorch. I have provided a bunch of references below if you are interested in practising and learning more.

I would like to thank Laurence Moroney for his excellent tutorial which I used as an inspiration for this tutorial.

Exercises 📚

Add more examples in the input and output tensors. In addition, try to change the dimensions of the data, say by adding an extra value in each array. What needs to be changed to successfully train the network with the new data?
The model converged really fast, which means it learned the relationship between x and y values after just a couple of iterations. Do you think it makes sense to continue training the model after the loss doesn’t keep decreasing any further? How would you automate the process of stopping the training after the model loss doesn’t substantially change?
In our example, we used a single hidden layer. Try to take a look at the PyTorch documentation to figure out what you need to do to get a model with more layers. What happens if you add more hidden layers?
We did not discuss the learning rate (lr-0.01) and the optimizer choice in great detail. Check out the PyTorch documentation to learn more about what other optimizers you can use. How do you think this affects training?