Kumparan Love Google Cloud Platform
Kumparan x Google

How To Use Google AI Platform Notebooks For Your Data Science Team

Bayu Aldi Yansyah
Dec 23, 2020 · 4 min read

In this article, I want to share my personal experience about how to use Google AI Platform Notebooks to reduce cost and increase the data scientist productivity.

Background

Jupyter Notebook is one of my team’s essentials tools. 90% of the projects starts with a prototype in a Jupyter Notebook .

Previously, we run a single Jupyter Notebook server to accommodate our need.

This approach come with a few drawbacks: (1) We can’t run multiple training process in paralel due to resource limitation (2) The backup procedure is complicated (3) Maintaining and hardening the security of jupyter notebook server is daunting task.

Using Google AI Platform Notebooks, each person in my team can easily spawn new instance based on the project requirements.

To reduce the cost, instance is temporary, it can be deleted without data loss. All notebooks data are stored in gitlab repository. For large data, we store it in Google Cloud Storage.

A Guide

These are the step-by-step to use Google AI Platform Notebook.

Create new git repository on Github or Gitlab. We will use it to store all your team notebooks. You can also use existing git repository in your organization.

Go to the Google AI Platform dashboard.

Choose Notebooks in the sidebar.

Click New Instance

Choose the instance type based on the project requirements

Follow the instruction and launch the instance.

Click Open Jupyterlab

You will redirected to unique URL like the following:

https://RANDOM_STRING-dot-asia-southeast1.notebooks.googleusercontent.com/lab

Don’t worry, it’s secure by default.

Only authenticated user can access the Jupyter Lab.

The next step is to setup the Git repository.

Select GitClone Repository to clone the git repository.

Click Clone a Repository.

Enter the gitlab/github URL.

If you are using https then enter your gitlab/github account email & password.

For ssh you need to setup the SSH first.

You will have access to your repository inside the file explorer:

Then create new notebook and work on your projects.

To backup your notebooks, see below.

Select Git, then pull the latest changes first.

See Untracked, for example:

Track your notebook file with hover the filename and click “+” icon.

Ignore the files that you don’t want to backup.

Write the commit message then, Click Commit.

If you have finished the projects, it’s best practice to delete your instance immediately.

All notebooks are stored in Git and can be used for any instances. So, don’t worry you can spawn new instance and use existing notebook.

Conclusion

Using approach that we described above, you will only pay what you use and your experiments/prototype are safely stored in Git repository.

You can easily spawn new instance and use existing notebook in the Git repository.

Good Luck.

Google Cloud - Community

Google Cloud community articles and blogs

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Bayu Aldi Yansyah

Written by

Lead Data Scientist @ Kumpran. Mostly write about applied Machine Learning

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store