How to Use Google Cloud and GPU Build Simple Deep Learning Environment

Create a Deep learning VM instance in Google Cloud Platform, install and set up Jupyter Notebook, and the Nvidia CUDA toolkit.

Jason Zhang
Nov 19, 2020 · 6 min read
Photo by on Unsplash

Google Cloud Platform provides us with a wealth of resources to support data science, deep learning, and AI projects. Now all we need to care about is how to design and train models, and the platform manages the rest tasks.

In current pandemic environment, the entire process of an AI project from design, coding to deployment, can be done remotely on the Cloud Platform.

I will demonstrate how to use Google Cloud Platform with GPU to build a deep learning environment from below four steps:

  1. Create a VM instance with GPU
  2. Set up Networking
  3. Install Juypter Notebook
  4. Install Nvidia CUDA

Step 1: Create and Starting a VM instance

Increase GPUs Quota

IMPORTANT: If you get the following notification when you create a VM that contains GPUs. You need to increase your GPU quota.

From top left menu, select IAM & Admin -> Quotas

In filter table, select Limit name and GPUs(all regions)

GPUs Quota Page

It will list your GPUs quota information, click ALL QUOTAS into Quota metric details page.

Checked Global option and click EDIT QUOTAS.

Enter the number of GPU quotas you need in New limit, I entered 2 here. Finally, click SUBMIT REQUEST.

Edit GPUs Quota Limits

Quota increase requests typically take two business days to process. Google will update progress information via email.

Create a VM instance

  1. In the Google Cloud Platform Console, go to the Home ->Computer Engine -> VM instances page
  2. Select your project and click Continue.
  3. Click the Create button.
  4. Specify a Name for your instance.
  5. Change the Region and Zone for this instance. I choose us-west1(oregon)and us-west1-b. Select a Machine configuration for your instance. In Series section, we choose N1 machine types that power by Intel Skylake CPU platform. There’s another N2 machine type, and while it uses the higher performance Intel Cascade Lake CPU palreform, the N2 do not support GPUs now, so we choose to use N1. In Machine type section, we choose n2-standard-8(8vCPU, 32GB memory).
  6. Expand the CPU platform and GPU at the bottom of the Machine configuration and add the GPU for your VM. Click Add GPU, Choose GPU type and Number of GPUs. To demonstrate I have chosen 2 pieces of NVIDIA Tesla K80 here.
    IMPORTANT: If you get the error Quota 'GPUS_ALL_REGIONS' exceed. Limit: 0.0 globally when creating a VM, you need to request a quota increase for your GPUs. Please see the previous section Increase GPUs Quota.
  7. In the Boot disk section, click Change to configure your boot disk.
    For Deep learning project, I choose Deep Learning on Linux in Operating System section.
    In Version section, we chose GPU Optimized Debian m32 (with CUDA 10.0) which is A Debian 9 based image with CUDA/CuDNN/NCCL pre-installed.
    In Size section, we need to choose at least 300GB.
  8. In Access scopes section, we check Allow full access to all Cloud APIs, because we need access to Google Bucket and other Cloud APIs. In Firewall section, we check both Allow HTTP traffic and All HTTPS traffic so that we can access the Jupyter Notebook from external network.
  9. Click on Management,security,disks,networking,sole tenancy and chose the Disks tab. In Deletion rule, unchecked Delete boot disk when instance is deleted. When you accidentally delete the VM instance, it will not delete the boot disk.
  10. Other sections can use the default value.
  11. Click CREATE, after waiting a few minutes, your VM is available.
Choose GPUs
Boot disk Section
Firewall Setting
A VM instance created

STEP 2: Set up Networking

In order to be able to access your Jupyter Note from the external network, you need to set up a static IP and a firewall rule in your new VM.

External IP addresses

  1. Go to Navigation menu -> NETWORKING -> VPC network -> External IP addresses
  2. Change your VM External address’s Type from Ephemeral to Static, and add Name for the new static IP address.


  1. Go to VPC network -> Firewall, Click CREATE FIREWALL to create a firewall rule for your Jupyte Note.
  2. In Create a firewall rule page, input Name of the rule, change Targets to All instances in the network.
  3. In Source IP ranges, input
  4. In Protocols and ports, checked Specified protocols and ports and set tcp port to 5000 or other port number.
  5. Leave others as default and click CREATE.
Create a firewall rule for Jupyter note
A jupyernote rule created

STEP 3: Install Jupyter Notebook

Back to your VM instances page and Activate Cloud Shell on the top right.

Install Jupyter Notebook using pip

pip install jupyter

If you get an error message about permission denied, use the command below.

pip install jupyter --user

Generate a configuration file:

$ jupyter notebook --generate-config

Go to Jupyter installation directory

$ cd ~/.jupyter/$ vim

Open , and add the following to the end of the file. Make sure you replace the port number with the one you allowed firewall access to above.

c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 5000

Launching Jupyter Notebook.Using the following command in your VM SSH window:

$ jupyter notebook

As shown below, your Jupyter Notebook is already running.

Open your browser and input the following address:

http://<External Static IP Addrsss>:<Port NUmber>

Sometimes you need to enter a token, copy the token given in the command line as shown above.

Once everything is set up, your Jupyter Notebook looks like this.


If you use Deep Learning On Linux public images like me, then Google cloud will pre-installed PyTorch, Tensorflow, CUDA etc.

Sometimes, to avoid unknown errors, I recommend you manually install CUDA.

Download CUDA

Tensorflow and PyTorch only support CUDA 10, we need to download CUDA Toolkit 10.1.

Choose the right target platform based on your Machine information.

IMPORTANT: In Installer Type section, we choose runfile(local).

Copy the installation command.

IMPORTANT: To avoid GUI errors, we should add the --no-opengl-libs option to prevent the openGL libraries from being installed.

Run the command below.

sudo sh --no-opengl-libs

Congratulation, you’ve got a simple deep learning environment. It’s time to train your model.

If you want to know how to use Goolge Cloud and TPU, please read this article.

Take care, and stay all safe and healthy.

The Startup

Get smarter at building your thing. Join The Startup’s +724K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store