Using API keys (and other sensitive data) in a Colab notebook

Colab notebooks are an amazing platform for prototyping and developing code. You don’t need to spin up resources, can quickly iterate and run code, and easily share notebooks for collaboration.

But what happens when your notebook code contains sensitive information like API keys or passwords? Rather than hard-coding this information, you can use Google Secret Manager to securely store and manage these secrets and then access them directly from Colab using a few lines of code, without exposing the underlying values.

In this post, I will share how to create a secret in Secret Manager and access that secret from a Colab Enterprise notebook.

Importing my API key from Secret Manager into a Colab Enterprise notebook.

I’ve started using Google Secret Manager anytime I use API keys in my notebooks for a few reasons:

• Security: It offers a secure way to protect the key from accidental exposure, especially when sharing my notebook or pushing the code to Github.
• Team Collaboration: It allows me to provide my teammates access to the secret using IAM so they can seamlessly run the notebook and use the same API key while authenticating through their own Google Cloud account.
• Consistency: Colab Secrets, while a convenient alternative, isn’t available in Colab Enterprise (which I often use) and doesn’t offer all the features of Secret Manager, including sharing of secrets across users or when using other Google Cloud products. It’s easier for me to stick to one — but your preference may vary!

Creating a Secret in Secret Manager

First of all, you’ll need a Google Cloud project created with billing enabled. Next you can create a secret in Secret Manager, using the following steps:

1. Go to the Secret Manager page and click on the “Create secret” button.
2. Enter a name for your secret. In the example screenshot below, I named the secret maps-api-key.
3. Add the value of the secret (such as the API key) directly in the “Secret value”, or upload a file containing the secret’s value using the “Browse” button.
4. Keep the rest of the options in their default values
5. Click on the “Create secret” button.

Screenshot of the Create Secret page with my API key secret value obscured

Accessing a Secret from a Colab Enterprise Notebook

To access a secret from a Colab Enterprise notebook, you’ll need to install the python library:

!pip install google-cloud-secret-manager

Then you can use the following code block to access the secret. Make sure you set the PROJECT_ID and YOUR_SECRET_NAME variables or adjust the secret_path code line appropriately.

import os
from google.cloud import secretmanager

# Assign project id and secret variables
PROJECT_ID = "YOUR_PROJECT_ID"
SECRET_NAME = "YOUR_SECRET_NAME" 

# Create a Secret Manager client
client = secretmanager.SecretManagerServiceClient()

# Access the secret
secret_path = f"projects/{PROJECT_ID}/secrets/{SECRET_NAME}/versions/latest"
response = client.access_secret_version(request={"name": secret_path})
secret_value = response.payload.data.decode("UTF-8")

# Set the secret value as an environment variable
os.environ["GOOGLE_MAPS_API_KEY"] = secret_value

Once you have accessed a secret from a Colab Enterprise notebook and set it as a variable, you can use that variable in your notebook code. You can give permission to other Google Cloud users to use the secret by granting them the Secret Manager Secret Accessor role on that particular secret.

Next Steps

Now that you’ve learned about Secret Manager, you’re equipped to keep your sensitive information safe and sound in your Colab notebooks. Want to dive deeper into best practices for API keys, Secret Manager, or Colab Enterprise? Check out these resources:

• Best practices for API keys
• Secret Manager
• Colab and Colab Enterprise

Feel free to share your experiences and ask any questions in the comments!