Managing Secrets for GCP Cloud Functions

A straightforward and secure solution for using keys in your Cloud Functions

Published in

Google Cloud - Community

7 min readNov 7, 2023

Analytics Engineering often involves building custom integrations between systems using Python scripts. In GCP, an easy way to automate such integrations with Python is by using Cloud Functions. Integrations often need credentials to access a target system and it is not uncommon to find hard-coded keys in the repositories of data teams. Sadly, there is often more drive to finish something fast than to figure out how to do it safely. I am writing this article because I believe that this topic deserves more attention in Analytics teams. Actually, you can set up a proper secret management system for your team in minutes in GCP, so there is no excuse not to!

First, I will describe common situations that increase the risk of keys getting exposed. Next, I will walk through the setup of a solution using Secret Manager. For each step, I will show you how to do it in the UI as well as from the terminal.

Let’s jump into it!

API Keys & Exposure Risks

An API key is a code or token that is used by software applications to authenticate and identify themselves when interacting with an API (Application Programming Interface).

When working in Python, there are a couple of (good and bad) ways to work with keys. For more background on this topic, I suggest reading this article. I will describe the most common ways in the next paragraphs.

Main Risk 1 — Hardcoded in Source or Git History

It’s not uncommon to find hardcoded unencrypted keys in the Git history, either in code or in a separate key file. Keys can be hidden in old commits, without showing up in the actual source code, this makes it difficult to spot without using special tooling. GitHub and BitBucket both have built-in secret scanning tools, that scan your entire git history for exposed secrets.

Example of a hardcoded key in your Git history (Image by Author)

Main Risk 2 — Terminal Command History

Each time you run a command in your terminal, it gets saved to your command line history. Someone who has access to the machine can read this history and find your key. Try the command history :

Example of an exposed key, defined in plain text. (Image by Author)

If you recognize above mentioned examples, it is time to take action. In the next chapter, I will demonstrate a solution.

Solution: Secret Manager

When your organization is using GCP, a way to improve protection and prevent the abovementioned exposures, is to use Google’s Secret Manager. You can use this tool to store and retrieve secrets, both in the cloud as well as on your local machine. I will show you a simple example of how to properly set up a Cloud Function that uses a secret, managed in the Secret Manager. I will set this up using the UI, and subsequently by using the gcloud SDK as well. The cloud SDK is a toolset that allows you to run commands in GCP from your terminal. For instructions on how to set up gcloud please refer to the official documentation.

Secret

Let’s set up a dummy secret in Secret Manager, through the UI:

You can similarly create a secret and set it’s value with this command:

gcloud secrets create demo_secret \
    --replication-policy="automatic" && \
gcloud secrets versions add demo_secret --data-file="/path/to/file.txt"

Note
Even though it’s possible with gcloud, avoid using plain text to define the secret version, because this will expose the value in the command line history as I mentioned earlier.

If you have created the secret correctly it will show up in your Secret Manager UI:

Result: A demo secret in your Secret Manager (Image by Author)

Service Account

Service Accounts can be used to divide permissions over virtual accounts. They allow you to implement the “Principle of Least Privilege” within your GCP Project. This is a crucial defense line protecting your environment. Google wrote a good article about this, specifically for Cloud Functions.

While by default most Google services (e.g. Cloud Run or Cloud Function) have their own service account already, it is a good idea to further separate permissions with service accounts for each group of Cloud Functions with a similar Permission need. See the image below.

With Service Accounts, you can prevent one application from having redundant access. (Image by Author)

Let’s create a new service account for our demo in the IAM UI.

Click “Create Service Account”, and fill in the details.

Setting up a service account (Image by Author)

You can also create a service account from the terminal with gcloud :

gcloud iam service-accounts create demo-service-account \
    --description="A demo Service Account" \
    --display-name="demo-service-account"

Finally, you need to allow your new service account to retrieve the demo_secret . Go to your secret, go to the Permissions tab, and assign the Secret Manager Secret Accessor role to your new service account.

To do the same from the terminal, type:

gcloud secrets add-iam-policy-binding demo_secret \
    --member="serviceAccount:demo-service-account@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/secretmanager.secretAccessor"

Read more on giving and revoking secret access here.

Cloud Functions

Next, we need a Cloud Function. Cloud Functions is a service from Google that allows you to execute simple scripts in a fully managed environment.

When you want to run a Python function the only thing you need to do is select a runtime environment (e.g. Python 3.11), define a requirements.txt and Google will spin up a Cloud Run Container with everything your script needs.

For more information on Cloud Run and Containerization please refer to my article on Containerization of a Python App. The main benefit of using Cloud Functions over Cloud Run is the simplicity (traded for less flexibility). For a detailed comparison read this article.

Let’s set up a Cloud Function that retrieves our demo_secret from the Secret Manager and prints it. Note that the UI allows two methods for adding secrets.

Mounted as Volume
Exposed as Environment Variable

Both options work, but as discussed before if someone confiscates the container, they could extract the files and environment variables.

The safest way to use a key in a cloud function is to retrieve it directly in the code from the Google Secret Manager API.

This means it is not stored as an environment variable or file in your Cloud Function instance.

To create a Cloud Function, click Create Function in the UI. Don’t set a new secret reference in the security tab. Just make sure you assign the service account that can access the demo_secret.

Add this code to the cloud function:

import functions_framework
# Import Secret Manager
from google.cloud import secretmanager


@functions_framework.http
def hello_secret(request):
  # Set variables
  project_id = 'YOUR_PROJECT' #Fill in your project
  secret_id = 'demo_secret'
  version_id = 'latest'  
  
  # Create Sercet Version Reference
  name = f"projects/{project_id}/secrets/{secret_id}/versions/{version_id}"

  # Instantiate Secret Manager Client
  client = secretmanager.SecretManagerServiceClient()
  
  # Get Response
  response = client.access_secret_version(request={"name": name})

  # Extract 'Plain Text' Key
  secret_key = response.payload.data.decode("UTF-8")

  return 'Hello you successfully called a key from Secret Manager: {}!'.format(secret_key)

And make sure to add the Google Secret Manager Python package to your requirements.txt :

functions-framework==3.*
google-cloud-secret-manager

When you test the Cloud Function, you will see this:

Response of the cloud function. **Note: Never print a real key like this.** (Image by Author)

If you want to retrieve the key on your local computer, authorize the terminal before you run your Python script:

gcloud auth application-default login

On top of that: Make sure your personal Google account has accessor permission to the secret you want to access, or impersonate the service account that does have access with:

gcloud auth application-default login --impersonate-service-account SERVICE_ACCT_EMAIL

This will allow you and your team to develop cloud functions locally, in a safe way, without physically storing key files on your computer.

Conclusion

That’s it. I hope you found this article useful, and are motivated to implement a safe approach to secret management within your team. You should now be able to start using GCP’s Secret Manager, in combination with your cloud functions (and other services). If you are interested in reading more on the topic of securing your GCP project, I suggest the following sources:

Enjoyed this Article?

If you found this article helpful, make sure to follow me to stay up-to-date with my latest articles on data modeling, coding, GCP, and more.