Develop with Google Colab and Google Drive

Stavros Filippidis
Sep 7, 2018 · 4 min read

I’m currently working on a Machine Learning Project at the University. As many ML Projects it involves the need for a good amount of storage and computational power. That together with the fact that i want to work on different devices and sometimes need to share my work with my adviser at the Uni my local computer is not the best option.

I started looking for alternatives and found an interesting service offered by Google called Colaboratory.

The free cloud service offers enough computation power through GPU support (for free) and a stable and well designed Jupyter Notebook environment.

How to use it?

Just go to Google Colaboratory and upload your Notebook from Google Drive, Github or from your computer.

Google Drive offers some pretty nice things:

These both two work together pretty well. Open for instance your notebooks from the Cloud like shown below.

Loading Data

There are two options to load data into your Google Colab Notebook. You can either upload it directly from your local storage or you can connect to Google Drive with your account and pull data from there.

Now be aware that your data and code/product is uploaded to Google Servers. Decide for your own what you want to share. You might also want to use Colab for testing purposes only.


Let’s jump in!

Create a Colab Notebook in your Google Drive and open it.

Install package

First install all the necessary frameworks.

And yes it is possible to install packages via your Notebook. Be aware though that although some libraries are already installed, some like PyDrive have to be reinstalled every time you open a new session.

!pip install -U -q PyDrive

Imports

PyDrive let’s us do all the data exchange and access Google Drive. This article will give you some insight into the functionality that PyDrive offers. For further details checkout the PyDrive Documentation.

Authentication

Before even interacting with your Drive Account, Google needs to know who you are. Run the code below to authenticate.

You will be explicitly asked for permission. Follow the link and copy & paste the verification code in order to proceed.

Overview

Now that you are logged in and have all the permissions let’s get an overview. In order to navigate through or access Drive data/files we need to work with Id’s. We first start at the root and loop through all the objects to print their title and id.

When we iterate through the file list we print out the file id's which we will use later to navigate to directories or access files.

This is how it works:

First the method which allows to list our files.

Define the directory from whom we will list the files.

In case everything worked your output should be as follows:

Title: MyGoogleDriveText.txt, id: sG02ptGX*************

Title: Directory_1 , id: PYU5tNFp3aDh0Yjg******

depending on directories and files. Let’s say we want to output all the filenames and content of Directory_1. Use the id:

Load your stuff to the notebook

We know have all the id’s to the corresponding files and want to upload them to our Notebook. We need to provide two parameters. The google drive id of the file we want loaded and the local environment name with which we will access the data later.

If you want to check if the data was loaded properly check the left menu in your Notebook.

You might ask why we go through all this process when there is an upload option in the side menu. This option allows only an upload from your local device.

Loading Directories

In case you want to upload a folder with images or some sort of other data you first need to zip it. Then upload it to the Notebook and unzip it. Uploading it unzipped did not work for me.

Work with your data.

In order to access your data simply use the local environment name as path inside the method you want to access/use your stuff. Here we print the first ten rows of our csv file with islice from itertools.

Save it back to the Cloud

You finished all your work and want to save the data. The following code snippet illustrates how to do it. In this example we will upload some sample data provided by Google Colab. Feel free though to upload any data by simply using their notebook name.

In case you want to play around a bit more click here and get the repository with the latest Notebook.

Have fun developing with Google Drive and Colab!

Thank your reading my first article on Medium, i hope you enjoyed and can use the stuff you learned today. If you have any questions, feedback or ideas for a new article simply text me on Medium.

Stavros Filippidis

Written by

I’m a computer science Student from Munich.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade