Introducing Google Cloud Logging Python v3.0.0

Manage your app’s Python logs and related metadata using Google Cloud

We’re excited to announce the release of a major update to the Google Cloud Python logging library!

v3.0.0 makes it even easier for Python developers to send and read logs from Google Cloud, providing real-time insight into what is happening in your application. If you’re a Python developer working with Google Cloud, now is a great time to try out Cloud Logging!

If you're unfamiliar with the google-cloud-logging library, getting started is simple. First, download the library using pip:

$ pip install "google-cloud-logging>=3.0.0"

Now, you can set up the client library to work with Python’s built-in `logging` library. Doing this will make it so that all your standard Python log statements will start sending data to Google Cloud:

# set up the Google Cloud Logging python client library
import google.cloud.logging
client = google.cloud.logging.Client()
client.setup_logging()
# use Python’s standard logging library to send logs to GCP
import logging
logging.warning(“Hello World”)

We recommend using the standard Python logging interface for log creation, as demonstrated above. However, if you need access to other Google Cloud Logging features (reading logs, managing log sinks, etc), you can use google.cloud.logging directly:

import google.cloud.logging
client = google.cloud.logging.Client()
logger = client.logger(name="log_id")
# read logs from GCP
client.list_entries(max_size=5)
# write log to GCP
logger.log("hello world", resource={"type":"global", "labels":{}})

Here are some of the main features of the new release:

Support More Cloud Environments

Previous versions of google-cloud-logging supported only App Engine and Kubernetes Engine. Users reported that the library would occasionally drop logs on serverless environments like Cloud Run and Cloud Functions. This was because the library would send logs in batches over the network. When serverless environments spin down, unsent batches could be lost.

v3.0.0 fixes this issue by making use of GCP’s built in structured JSON logging feature (GKE, Cloud Run, or Cloud Functions). If the library detects it is running on an environment that supports structured logging, it will automatically make use of the new StructuredLogHandler, which writes logs as JSON strings printed to standard out. Google Cloud’s built-in agents will then parse the logs and deliver them to Cloud Logging, even if the code that produced the logs has spun down.

Structured Logging is more reliable on serverless environments, and it allows us to support all major GCP compute environments in v3.0.0. Still, if you would prefer to send logs over the network as before, you can manually set up the library with a CloudLoggingHandler instance:

from google.cloud.logging.handlers import CloudLoggingHandler
from google.cloud.logging_v2.handlers import setup_logging
# explicitly set up a CloudLoggingHandler to send logs over the network
handler = CloudLoggingHandler(client)
setup_logging(handler)
import logging
logging.warning(“Hello World”)

Metadata Auto-detection

When you troubleshoot your application, it can be useful to have as much information about the environment as possible captured in your application logs. google-cloud-logging attempts to help in this process by detecting and attaching metadata about your environment to each log message. The following fields are currently supported:

• resource: The Google Cloud resource the log originated from
  – For example, Functions, GKE, or Cloud Run
• httpRequest: Information about an HTTP request in the log’s context
  – Flask and Django are currently supported
• sourceLocation: File, line, and function names
• trace, spanId, and traceSampled: Cloud Trace metadata
  – Supports X-Cloud-Trace-Context and w3c transparent trace formats

The library will make an attempt to populate this data whenever possible, but any of these fields can also be explicitly set by developers using the library:

logging.info("hello", extra={
    "labels": {"foo": "bar"},
    "http_request": {"requestUrl": "localhost"},
    "trace": "01234"
})

JSON Support in Standard Library Integration

Google Cloud Logging supports both string and JSON payloads for LogEntries, but up until now, the Python standard library integration could only send logs with string payloads.

In `google-cloud-logging` v3.0.0, you can log JSON data in two ways:

1. Log a JSON-parsable string:

import logging
import json
data_dict = {"hello": "world"}
logging.info(json.dumps(data_dict))

1. Pass a `json_fields` dictionary using Python logging’s `extra` argument:

import logging
data_dict = {"hello": "world"}
logging.info("message field", extra={"json_fields": data_dict})

Next Steps

• For more information on the latest release, check out the full v3.0.0 Migration Guide
• For developer documentation, see the google-cloud-logging user guide
• If you have any feedback about the latest release, or would like to make any contributions, feel free to open issues on our GitHub repo
• If you want to learn best practices of instrumenting your code for observability and running workloads reliably on GCP, try Cloud Ops Sandbox