Serverless Computing Made Easy

Google Cloud Platform Cloud Functions are a convenient way to go serverless

The Google Cloud Platform (GCP) has a powerful serverless computing feature. Cloud Functions not only allow functional programming on-demand, they can server as cloud-wide event handlers.

Narrowing our discussion down to a common use-case, we can trigger Cloud Functions from events logged in Stackdriver. In this case, we will use an event which we can easily control; attaching an external IP to a virtual machine. When a static IP change is detected, the affected VM is migrated within the GCP environment. The practical purpose of this is to repair a VM that is failing to register its new static IP address.

Attaching static external IPs to a VM creates a Stackdriver logging event

The source code for this blog post is available here on Github. An install script is provided (gcloud_script.sh), along with instructions in the README file. The install script will configure all of the necessary GCP resources for readers to follow along. Please be aware of the Linux shell scripting variables at the beginning of the install script. You need to at least PROJECT_ID to your own GCP project.

Source code available on Github (click here)

Looking through the installation script, there are a series of gcloud commands. The application of these commands are useful in general, for a GCP toolkit. Of specific interest are the commands that create the logging sink and the GCP Cloud function:

The logging sink connects Stackdriver to Pub/Sub

The Cloud Function is triggered by the Pub/Sub topic, written to by the logging sink

The logging sink is what actually wires everything up for the Cloud Function. A Stackdriver log filter captures matching logging entries, in real time. These entries are then captured by the logging sink, which then publishes to the Pub/Sub topic. A deep discussion on Pub/Sub is beyond the scope of this blog post, which is why an installation script is provided. Likewise, the install script creates the Cloud Function.

The Stackdriver logging sink’s destination is a Pub/Sub topic, visible in the GCP console

The Cloud Function is a snippet of code that exists on a temporarily available VM resource. This is not a VM that a human user controls, nor is it visible in the GCP console. What is visible is the special Google Cloud App Engine service account that executes the Cloud Function.

The App Engine service account executes Cloud Functions on the user’s behalf

The body of the Cloud Function is contained in main.py in the Github repository. This name is important as gcloud looks for that specific name, when creating the Cloud Function itself. gcloud also uploads main.py into the Cloud Function workspace in your GCP project. This function is then given the name you assigned it. Although the mechanics of Cloud Function creation are beyond the scope of this article, what is important is the creation syntax. Using the syntax in the install script is a reliable way of creating Cloud Functions. Other methods exist, but they often provide non-descriptive errors.

main.py contains the Cloud Function body, in this case written in Python

Taking a closer look at the Cloud Function body, the most important step is line 8. Here the Cloud Function authenticates against Google Cloud. Since the App Engine service account already has the necessary permissions, this step is parsimonious. By comparison, if we were writing a script to execute outside of a Cloud Function, authentication would require more labor.

Line 11 accounts for Stackdriver logs being base64 encoded. Line 12 then accounts for Stackdriver logs being kept in JSON format. Lines 15 to 17 access various elements in the JSON tree, with each set of braces representing tree depth.

On the Stackdriver Logging page in Cloud console, click the little triangle and go to advanced filter mode

Enter the advanced log filter from the gcloud_script.sh installation file (remove the wrapping quotes)

With resource being the JSON root node, labels and project_id are one and two levels deep

Line 20 in the Cloud Function is what triggers the VM migration.

Understanding all of this, a static IP can be attached to a VM to trigger the Cloud Function. Two static IPs were created by the gcloud_script.sh installation script, static-ip-1 and static-ip-2. Assigning either to one of the script-created VMs will trigger the Cloud Function.

In the VPC network, External IP addresses section of the GCP console, a static IP can be manually assigned to a VM

Cloud Function invoked, as shown as a spike on the invocation graph

Finally, the Cloud Function itself creates a Stackdriver event. This is useful for confirming that the Cloud Function actually worked. A log entry containing “GCE_OPERATION_DONE”, for the App Engine service account will provide this confirmation. For convenience, the Stackdriver logging filters used in this blog post are also in the Github repository.

NOTE: There is occasionally a lag between Cloud Function execution and its logging in Stackdriver. If you suspect there’s a legitimate bug, scroll down for a link to the debugging article.

The next article covers debugging serverless functions, which will empower you to go serverless with confidence.