On a previous episode of AI Adventures, we looked at Colab as a great way to get started in the data science and machine learning world. But there are some models out there that need to run for a long time (Colab instances will reset after several hours), you want to get more memory or GPUs than is provided for free.
The question then becomes: How can we hook up Colab’s frontend with some more compute power? We’re going to use Google Cloud Platform’s Deep Learning VMs to power up your Colab environment. A while back, we looked at how to create Deep Learning VMs with your choice of machine learning frameworks. We’ll use that example today and connect Colab to it so we can utilize the resources on that machine.
If you want to catch this episode in video form, I’ve linked it here:
To start out, we’ll of course need to make our VM, and let’s make it a big one! Head over to the Cloud Marketplace and find the Deep Learning VM and select it. I’m going to call minecolab-vm. And for good measure let’s include 16 CPUs, with 60 GB of memory! And for the GPUs, I’ll choose the V100s, and put in two of them! Now we’re really cooking!
While we’re waiting for that to spin up and launch JupyterLab, let’s talk a bit about how we’re going to get Colab to talk to your VM. There are 2 tricks that we are going to use, which together make this connection possible. The first is the fact that Colab can connect to a local runtime. So if you have a local Jupyter notebook server running, you can use Colab as the frontend to that server, rather than the usual Jupyter Notebook frontend. You can access this option by clicking the drop-down menu in the upper right-hand corner of Colab and selecting “Connect to local runtime”.
The second trick we’ll take advantage of is that we can do port forwarding on the Deep Learning VM we just created, so that it can be accessed from our local machine. You can do it with this command (just swap out colab-vm for your VM’s name):
gcloud compute ssh colab-vm -- -L 8080:localhost:8080
Be sure to run it from your local terminal, not Cloud Shell. In this particular situation, since we wish to forward the port to our local machine, running the command in Cloud shell does not achieve our goals. Additionally, notice that the command features a double-dash: --. This command causes everything that comes after it to be sent to the “next layer” of the commands given, so to speak. In this case, it means that the -L 8080:localhost:8080 is given to the ssh command, rather than to gcloud compute. It configures what the mapping of the ports should be from the virtual machine to your local machine.
Just having the port-forwarding on its own would just mean we can access the Jupyter server from a local frontend, which you can do by going to http://localhost:8080. But if we open up Colab and tell it to connect to our “local” runtime, the local runtime it will see is actually the port-forwarded one from the Deep Learning VM! And thus the two become connected.
On my V100-powered Colab instance, I can run nvidia-smi to confirm that yes indeed, we have our two V100 GPUs powering this instance:
So now you understand how Colab and Deep Learning VMs can work together, and how to get your setup just how you want it as your needs change, both in terms of what the frontend looks like, and what backend system is driving it. For example, you develop against a local backend initially, and when you want to get more power, you can connect it to your Deep Learning VM and run a big training job.
How do you set up your data science environment, and how do you allow it to change, flex, and grow as needed? Share your approach in the comments below!
For now, head over to the Deep Learning VMs and try connecting it with your Colab notebook!
Thanks for reading this episode of Cloud AI Adventures. If you’re enjoying the series, please let me know by clapping for the article. If you want more machine learning action, be sure to follow me on Medium or subscribe to the YouTube channel to catch future episodes as they come out. More episodes coming at you soon!
