Debugging Serverless Functions

With serverless computing, the real labor is in effective debugging

…continued from Serverless Computing Made Easy

Google Cloud Functions are a powerful way to implement serverless computing. Fully wielding this tool requires a reliable means of debugging code. In this blog post we work with a Python 3 Cloud Function, and walk through a reliable method for troubleshooting.

The example code for the below debugging exercises is available here on Github. This is the same repository used in the previous blog post. Also, it is assumed that you followed the installation steps, either from the blog post or the repo readme file. The Google Cloud Platform (GCP) resources created during installation will be referenced by the testing Cloud Function.

Source code and debugging setup instructions are available on Github (click here)

Referring to the repo readme file, the debugging installation instructions are as follows:

1. Clone this repo to your workspace

2. Navigate to the debugging folder

3. Edit the prepare_log.sh script, adding in your Google Cloud Platform (GCP) specific values for project ID, zone, region and virtual machine (VM) instance ID

Add your GCP values in prepare_log.sh, INSTANCE_ID is on the VM details page

The VM instance ID is a unique numerical identifier used in Stackdriver logs

The script using the sed command to swap in the variable values. A cleaner method would be to use envsubst, but I could not get this to work in my Ubuntu environment. It is worth trying, if you feel like writing your own log preparation script.

An important piece of enrichment is how the sample log data in sample_log.json was created. The process is to download an existing Stackdriver log that triggers your production Cloud Function:

Filter for the logs you want; a sample filter is provided in the Github repo, advanced_filter_static_ip_event.txt

Either “View in new tab” or “Download” to view the raw JSON text of the desired Stackdriver log entries

The next step in preparing your own testing logs is to remove the leading and trailing square brackets, “[“ and “]“. Finally, you need to minify the JSON, removing all new line characters. The Cloud Function tester breaks when new line characters are included. Please see debugging/sample_log.json in the Github repo, for an example of a clean sample log file (with shell variables present though).

4. Run prepare_log.sh, which will produce a file called prepared_log.json

Cloud Shell was used to clone the repo and create prepared_log.json, which his the GCP specific values

5. In the GCP console, create a Python 3 Cloud Function, the “Trigger” will be “Cloud Pub/Sub” and the “Topic” will be the one created in the above Installation steps; for “Source code” select “Inline editor” and for “Runtime” select “Python 3.x”

6. Copy-paste the contents of debugger/main.py into the “main.py” field

NOTE: Your terminal may line break prepared_log.json when you try to copy its content. If the Cloud Function “Testing” tab gives you errors, this is likely the reason. You will have to make the JSON text a single continuous string, with no line breaks, in order for the “Testing” tab to accept it.

7. Continue creating the Cloud Function, naming it and the “Function to execute” as test_migrate_vm

It is crucial that the Python def name match the “Function to execute” name

Notice lines 5, 12 and 13 in the Python code, also available in the Github repo as debugging/main.py. In line 5, we have to rename the defined function (and likewise the “Function to execute”. We do not want the testing Cloud Function to have a naming conflict with the production one.

Line 12 expects the Stackdriver log line directly from the Stackdriver logging sink. Stackdriver logs published to sink (or Pub/Sub topic) are Base64 encoded. However, our test data is plain text, and Python considers it to be a simple collection. Notice how the testing data is encased in curly braces, “{“ and “}”, with comma separated key-value string pairs. This makes the elements easy to reference, like in line 14.

8. Once the Cloud Function is created, navigate to the “Testing” sub-tab

9. Copy paste the contents of prepared_log.sh into the “Triggering event” text box

10. Click the “Test the function” button

11. On correct execution, the “Output” should be “OK” and logs should be generated

If the function ingested the test data and executed correctly then it outputs “OK”

12. The print statement at the end of the Python code will output to Stackdriver log. Print statements that you add, like print(“test_string”), are also output to Stackdriver. On the Stackdriver Logging page, make sure to select the “Cloud Function” filter dropdown resource, and then the Cloud Function name.

In Stackdriver Logging, the “Cloud Function” filter dropdown resource takes you to the output

Hopefully the above debugging methodology will help you troubleshoot your own Cloud Functions. Remember, the key is leaning on Stackdriver to provide you with test data input, and to capture your Cloud Function output. As well print statements can be used for debugging, with their output also captured in Stackdriver.