Go serverless: GCP Cloud Functions, explained

And how to build and (locally) test your own serverless Python functions

Photo by NASA on Unsplash

Google Cloud Platform offers all of the industry-standard tools for cloud processing, including storage and compute, and is my choice of cloud provider for data science tasks (namely because of ecosystem with other GCP tools such as BigQuery or Airflow that offer intuitive data engineering and MLOps tooling). It also offers an alternative to AWS Lambda, a service that offers the ability to host serverless functions, called Cloud Functions.

What does “serverless” mean?

“Serverless” doesn’t actually mean that your code runs without a server. In fact, you can run serverless functions on a local server (we’ll get to that in a moment). Serverless refers to the fact that you, as a client, do not have to handle putting up and taking down a server every time you want to run some of your code, and you definitely don’t have to handle maintenance or initialization of the server (the cloud provider handles that). Instead, you can write some code that runs every time a request is made to the server—most serverless compute providers have runtime limits of around 10 seconds, and occasionally they’ll allow asynchronous calls for a few minutes. This means they’re best suited for tasks that don’t need to store any data, only injest a POST request, do something with the data, then spin down after a few seconds.

Writing your own serverless functions

Cloud Functions supports Python, as well as several other standard languages, though it, like many other cloud serverless providers, also allow you to specify your own runtime. The rest of this tutorial will refer to Python, but you can easily extend the concepts to your programming language of choice.

To test locally, all you’ll need is a script called main.py in a new folder, in which you can define any function you want. The only stipulation is that the function must return something, ideally a JSON or a piece of text. For example, in main.py:

def say_hello(event):
    name = request.args.get('name', None)
    if name is None:
        return f"Hi there!"
    else:
        return f"Hi, {name}!"

where request is the Flask.request object.

Next, you’ll need to install functions-framework, the framework that GCP developed to run their Cloud Functions. In Python, it’s really just a wrapper for Flask that handles the request information of the POST request. You can install it by using pip install functions-framework.

Now run functions-framework --target=say_hello in your terminal. Your server is now running, likely on http://localhost:8080.

To make a POST request, either use curl or write a second Python script that you can run, like so:

import requests

params = {
    "name": "Neha"
}

print(requests.post("http://localhost:8080", json=params).content)

Later, on Cloud Functions, you’ll need a requirements.txt file. I recommend creating a new conda environment through conda create <env_name>, then conda activate <env_name>. This way, you can keep track of the modules you had to install for this project alone, as well as a particular Python runtime if you’d like to specify it. To export the requirements at the end, run pip list --format=freeze > requirements.txt.

Hosting on GCP Cloud Functions

You’ll need an active Google Cloud Platform account to use Cloud Functions. Once you’ve signed up, look up “cloud functions” in Products in your first project. Follow the steps to set up and deploy your first Cloud Function! You can select from multiple runtimes or use a custom runtime, and use the inline code editor or upload files as a .zip.

Take a note of a few permissions:

1. If you’d like your API to be visible to the public, make sure you have selected Allow all traffic under Edit > Runtime, build, connections and security settings. This ensures that all traffic can actually access your function.
2. If you’d like all traffic to be able to run your function, you’ll need to take care of one more permission: Go to the Permissions tab, then add a new user by hitting Grant Access. The name of the new user should be “allUsers.” Give them the “Cloud Functions Invoker” role.

Happy coding!

Neha Desaraju is a student at the University of Texas at Austin studying computer science. You can find her online at estaudere.github.io.