How to use Google Colab with GitHub via Google Drive

In this tutorial, we will be discussing on how Google Colab can be used with GitHub for our Data science, Machine Learning projects and also use Google Drive as a cloud data storage. Let me introduce these products.

1. Colaboratory, or Colab for short, is a product from Google Research. It allows anybody to write and execute arbitrary python code through the browser, and is especially well suited to machine learning, data analysis and education.
2. GitHub is a code hosting platform for version control and Collaboration. It lets you and others work together on projects from anywhere. Thus allowing seamless collaboration without compromising the integrity of the original project.
3. Google Drive provides file storage and synchronization service, which allows users to store files on their servers, synchronize files across devices, and share files. It offers 15 GB of free storage to users.

Below picture depicts how these product interact with each other,

Picture by Author — Interaction between GitHub, Colab and Drive

Step 1: Use colab notebook as a Shell

1. Visit Google Colaboratory website
2. Click on New Notebook button. A blank notebook is initialized and opened

Step 2: Mount Google Drive to Google Colab Notebook

• Run the below script to mount your Google Drive

from google.colab import drive
drive.mount('/content/drive')

Screenshot by Author — Authenticating Google account

• Click the link to authenticate user Google account
• Select the respective Google Drive account on which you want to mount and click on sign in
• Copy and Paste the authentication code into the input cell
• Congrats! Your Google Drive is mounted,

Screenshot by Author — Google Drive mounted

Step 3: Change present working directory

• Below shell command will set the present working directory to,
  /content/drive/MyDrive/Github

%cd /content/drive/MyDrive/Github/

Note: Your Google Drive’s Home directory is at, /content/drive/MyDrive/

Step 4: Generate GitHub Access Token

Now its time to generate your GitHub token, that can be used to access the GitHub API.

• Visit GitHub website and login to your account.
• Go to Settings, navigate to Developer settings and then click on Personal access tokens. Your page should look something like,

Screenshot by Author — https://github.com/settings/tokens

• Click on Generate new token button on top right corner of the page.
• Click the repo checkbox under Select scopes section as shown,

Screenshot by Author — Generating new access token

Learn more about scopes of access token.

• Now, click on Generate token button at the bottom of the page. Now, the page should look like,

Screenshot by Author — Generated access token

You have successfully generated access token for your GitHub account. We will be using this token to access GitHub API. Note: Do not share your access token to the public as I did, I’ll be using for the purpose of this tutorial only.

Now, there arises two different scenario.

1. Create a new git repository from scratch
2. Clone an existing git repository from GitHub

Based on your requirements, follow the below steps,

Step 5.A: Create a new Git repository

Follow the below steps to create a new git repository from scratch directly in your Google Drive,

Step 5.A.1: Initialize new Git repository

• Initialize git using git init <directory> . In this tutorial, we will be using titanic repository.
• Change your working directory to the created repository.
• List the files and folder using ls command.

Screenshot by Author — git init command

Step 5.A.2: Working with Git repository

• It’s time to add files and folders to your working directory.
• git status to view the state of the working directory and the staging area.
• git add to add changes in the working directory to the staging area.

Learn more about How to download Kaggle datasets to Google Colab.

Screenshot by Author — Adding files to working directory

• After adding files and folder as per your requirements, commit your work using git commit -m "message" .

Step 5.A.3: Create a new repository on GitHub

Once you are satisfied with your work and want to save your commits to GitHub, follow the below steps,

• Visit GitHub official website and login with your account.
• Create a new repository in GitHub. Note: Do not initialize the repository with README, .gitignore or license file. This empty repository will await your code. Don’t worry, you can create those files in Google colab (local machine) or after your first git push on GitHub.

Screenshot by Author — Create a new repository on GitHub

Step 5.A.4: Upload your commits to GitHub

Before pushing your commits to your GitHub account, you need to configure your Google Drive git repository. To do this follow the steps,

• Create a set of variables from your GitHub account,

https://github.com/<username>/<repository> 

1. username — Your GitHub username. In our case, its MohammedIsmailP.
2. repository — Created repository. In our case, its titanic.
3. git_token — Your personal access token (Do not share to public).

• Add remote to your git from the above variable as,

git remote add <remote-name> https://{git_token}@github.com/{username}/{repository}.git

• Push your commits using git push command as,

git push -u <remote-name> <branch-name>

Screenshot by Author — Pushing commits to GitHub

Congratulations!! You have successfully pushed your commits to GitHub. Let’s verify it in GitHub.

Screenshot by Author — git commits reflected in GitHub

Step 5.B: Clone an existing Git repository

Follow the below steps to clone an existing git repository from GitHub into your Google Drive,

• Go to your GitHub repository to clone the repository.
• Click on Code button and copy the url as shown,

Screenshot by Author — GitHub repository page

• You’ll need your GitHub access token before cloning your GitHub repository. Also extract set of variables from your GitHub account.

1. username — Your GitHub username. In our case, its MohammedIsmailP.
2. repository — Created repository. In our case, its titanic.
3. git_token — Your personal access token (Do not share to public).

git clone https://{git_token}@github.com/{username}/{repository}

Screenshot by Author — git clone with GitHub access token

Congratulations!! You have successfully cloned your GitHub repository. You can start working on your repository and save your works by committing them using git commit -m "message" .

Screenshot by Author — Committing changes to Git

Once you are satisfied with your work and want to save your commits to GitHub, you can simply push your commits to GitHub using git push command.

Screenshot by Author — Pushing commits to GitHub

Congratulations!! You have successfully pushed your commits to GitHub. You can verify them in GitHub.

Screenshot by Author — git commits reflected in GitHub

That’s it! Our tutorial ends here!! We have successfully learned ‘How to use Google Colab with GitHub via Google Drive’. I hope you enjoyed doing along with this tutorial. But wait, we never learned ‘How to run notebook stored in Google Drive’. Don’t worry, I’ll cover this as well.

How to run notebook stored in Google Drive

To run your notebook stored in Google Drive, follow the steps.

• Open your Google Drive and login with your account.
• Locate your notebook in Google Drive and right click on the notebook to open actions menu, hover over Open with, and select Google Colaboratory as shown,

Screenshot by Author — Open notebook from Google Drive to Google Colaboratory

• Desired Notebook will be opened in Google Colab in a new tab.
• Mount your Google Drive to Google Colab (refer step 2).
• Change the working directory (refer step 3).

Screenshot by Author — Mount Google Drive and Change working directory

• Then execute your code.

That’s it! Simple, isn’t it? Hope, it helps. Happy Learning!!

Reference

1. Scopes for OAuth Apps
2. How to download Kaggle datasets to Google Colab
3. Learn more about Git and GitHub from Git Handbook