5 tips for Cloud Logging that everyone should know
Logs are essential for reducing MTTD and MTTR, here are my top tips for improving your efficiency with Google Cloud Operations Logging.
JSON logs are automatically indexed
Google named this feature Structured logging.
All logs captured by Google are automatically parsed, if the log format is a valid json format, Google will automatically switch from text to structured log.
The following invalid json format is interpreted as a text log. A limited attribute list makes this log entry difficult to use.
A similar entry with valid json provides an extensive list of information that you can easily use with complex filters, summary row attributes, and more.
On structured logging, you should follow Google’s pattern. Some special fields can be set to fill in log severity, tracing context, or execution details.If you don’t follow this pattern, the attributes will show up under jsonPayload. . More details on https://cloud.google.com/logging/docs/structured-logging
Get notified based on log content
You can trigger alerts if logs match patterns. Imagine you are building a strong monitoring alert system based on structured logs, how do you ensure that all logs follow the correct format and trigger an alert if not?
Creating log-based alert policies will help you to detect unexpected log content. A structured log will greatly help you configure the logs to include in the alert.
The following example triggers an alert for each unstructured log in my project for a given application.
You can now extract values from your logs
Continuing the counter provided by log-based metrics and used by log-based alert policies, you can now create distribution metrics.
Distribution metrics helps you to extract values from your logs and save them to a histogram bucket. Values are not recorded individually, only the distribution between buckets is stored.
Don’t forget to inject the tracing context
One of my favorite Cloud Logging features, if you extract the tracing context from your server requests, you can inject this information into the log statements. A very useful add-on to track a particular execution.
You can do this easily using OpenTelemetry Logging Instrumentation by example.
Google pattern requires logging.googleapis.com/trace and logging.googleapis.com/spanId, not directly trace_id and span_id.
Configure retention using Logs routers and Logs Buckets
As your Cloud Logging usage increases, your costs also increase.
Log entries that aren’t excluded might result in charges. For more information, see Cloud Logging pricing.
https://cloud.google.com/logging/docs/routing/overview
If you keep all your logs with a high retention period, the costs will increase and you may delete the logs on the application side to optimize them, which is not a good approach (unless they don’t provide useful informations).
Instead, you can use logging sinks to automatically delete logs based on patterns or attributes or also move them to a dedicated bucket with a low retention period.
I hope my 5 tips will help you improve your use of Cloud Operations Logging!
There are more things you can find out about Cloud Operations Logging like log regionalization, centralization, access control and more.
Feel free to continue your Cloud logging journey with some best practices on Google’s YouTube channel.
