Colab + Vs Code + GitHub + Jupyter (Perfect for Deep Learning)

Setup Google Colab remote machine with Vs Code | Get your local repository onto Colab | Run Jupyter Notebook from within Colab using port forwarding

After reading this article, you will be able to achieve the following(I have shown the tutorial in Linux which can be adopted for other OS users too).

1. You won’t have to rely solely on the jupyter notebook hosted by Google Colab to access their GPUs, Internet Speed and other resources. You can access there resources straight from your Vs Code editor (which offers the advantage of being able to navigate repositories that include multiple folders, and interdependent code files). It will feel like working on your local machine using Vs Code navigation features, GUI etc while accessing the compute of Colab.
2. Push your local files to GitHub and clone them to Colab machine. Develop and work around with your files on Colab. At the end of the day save your work by pushing your updated repo back to GitHub.
3. Run Jupyter notebook from inside Colab that will be hosted on your local machine’s browser.

Connecting Vs Code with Google Colab

1. Download a Cloudflare binary file from here . Extract from the downloaded zip file and keep the binary file in any suitable directory of your choice. In my case the location for binary file was

“/home/<username>/Downloads/cloudflared”

Cloudflare binary file

1. Open Vs Code and download Remote-SSH Extension.

Remote-SSH Extension in Vs Code

3. Inside Vs Code use (Ctrl + Shift + P) and type Remote Configuration File .

Remote Configuration File Search

Now select the file named as

‘/home/<username>/.ssh/config’ 

and paste the following command inside the file.

Host *.trycloudflare.com
 HostName %h
 User root
 Port 22
 ProxyCommand <PUT_THE_CLOUDFLARE_FILE_PATH_HERE> access ssh --hostname %h

4. You are done setting up Vs Code to access Colab Machine. Now go to Google Colab and open a new notebook. To get access to GPU change the runtime type to GPU and run the following commands in the notebook.

!pip install colab_ssh --upgrade
from colab_ssh import launch_ssh_cloudflared, init_git_cloudflared
launch_ssh_cloudflared(password="mypassword")

This will land you to the following screen.

Google Colab with Cloudflare link (to be copied to Vs Code)

5. Copy the link under Vs Code Remote SSH. Open Vs Code and type (Ctrl + Shift + P) . Search for Remote: Connect to Host and press Enter. Paste the copied link when prompted to, as shown below and press Enter.

Search for Remote:Connect to Host

Pasting the Cloudflare URL

6. Provide the password which you had setup in colab. There you go, you will be able able to access the machine from Vs Code Terminal. Type nvidia-smi to check the GPU config.

Colab GPU accessed from Vs Code Terminal

We are done linking Colab with Vs Code ;) .

Some insights about workflow

Now that you can access Colab GPU from your Vs Code terminal, it gives you the same flexibility you have while using Vs Code in your local machine. For instance if you want to access files in your local repository containing your DL project and run them using Colab GPU, all you need to do is create a private repository in GitHub and push to it. You can then clone the private repo in the colab machine from your Vs Code terminal.

This gives you access to your files in colab and you can use Vs Code to navigate between them seamlessly using its GUI. After you have completed your work for the day, push your changes back to GitHub.

All your project files and folders accessible from Vs Code

Tip

Problem 1: Sometimes due to unstable internet it happens that we get disconnected from Colab and when we run the notebook again Colab gives us a different machine.

Solution :To ensure that we land up in the same machine even after disconnecting, just run an infinite loop in colab like while True : pass. Since colab isn’t inactive even after we got disconnected, we will be connected back to the same machine. Follow Step 4&5 to obtain a new link to connect Vs Code Editor back to the same colab machine.

Problem 2: Sometimes Vs Code disconnects from the remote colab machine and cannot communicate with it using the cloudflare generated link anymore.

Solution: Follow Step 4&5 to obtain a new link to connect back to colab.

Install miniconda on Colab | Setup terminal to recognize conda | Access jupyter notebook from within Colab

1. To install miniconda on colab and activate terminal so that it recognizes conda, run the following commands:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh -b
export PATH="/root/miniconda3/bin:${PATH}"
conda init

Restart the terminal and conda will be enabled for your colab machine.

2. Create and activate an environment to get started with your project, using the following commands.

conda create -y -n envname python=3.7
conda activate envname

3. Before installing anything make sure to install the google-colab package using the command mentioned here. The google-colab package is necessary to run jupyter notebook and installs a compatible ipykernel version too.

4. To activate your conda environment in Jupyter notebook, run the following command.

python -m ipykernel install --user --name=envname

5. Now finally run the command mentioned below to activate jupyter notebook.

jupyter notebook --ip 127.0.0.1 --port 8888

This command hosts the notebook on IP= 127.0.0.1, which is necessary because the default IP where jupyter notebook is generally hosted (127.0.0.0) is already being used to host the Google Colab Notebook. Vs Code automatically does port forwarding at back-end to map the port of colab machine to our local machine.

Now you should be able access your jupyter notebook from within colab ;) .

Thats all for this tutorial. Hope it comes in handy and makes deep learning more fun & hassle-free. Please drop a like to my first ever medium blog and comment for any queries ❤.