The Wondrous World of Cloud Audit Logs!
Almost every Google Cloud service is able to produce an event in your project known as a Cloud Audit Log. These logs keep a record of activity within your project. Example activities include: a new VM was created, an IAM role was modified, or a Cloud Storage bucket was accessed.
In this blogpost, we’re going to explore Cloud Audit Logs and show how you can use Eventarc to listen to these log streams, configure event filters, and trigger your Cloud Run service.
What are Cloud Audit Logs?
Cloud Audit Logs are the log streams for Admin Activity, Data Access, System Event, and Policy Denied events. These events are often used to help customers with their audit and compliance needs. They’ll answer the “who did what, when, and where” within your project. 🔗
Here are the audit log types:
1 Admin Activity audit logs contain log entries for API calls or other actions that modify a configuration or metadata. 🔗
2 Data Access audit logs record API calls that read resource configuration, metadata, or user-level API calls that read or write resource data. 🔗
3 System Event audit logs record system administrative information generated from Google systems — not driven by direct user action. 🔗
4 Policy Denied audit logs record unauthorized access attempts to a service, attempted by a service account or user — useful in some scenarios. 🔗
I personally find Data Access logs to contain useful event information. We’ll be referring to those logs in the rest of this blogpost.
Enabling Audit Logs
Before jumping in, we need to enable data access audit logs for the service that we want to produce audit events. We can enable events in the Audit Logs page under IAM and Admin:
You may enable audit logs for all services with the default config, although, I find it’s easier to find the log entries I want to observe by individually picking services.
Viewing Audit Logs
After you’ve enabled audit logs, your project will start producing and retaining these logs. By default, data access logs are retained for 30 days.
Logs Explorer
A great way to view logs is through the logs explorer. Here, you have a powerful query interface where we can view our data access logs:
When an action is performed in your project, you’ll see an audit log such as this one for creating a Cloud Function (some personal data is faked):
Here you can see a log entry with some useful fields in our protoPayload. Here are some highlights:
serviceName(always present): The API/service that was logged.methodName(always present): The action that was performed within the service.resourceName(sometimes present): The resource that was modified when performing the action.
In this case, we have the method (CreateFunction) logged with and details about the function (a function in secret-project, location us-central1, and function name of new-function).
Triggering Cloud Run services with Eventarc
Now that you have created an audit log, what can you do with that info?
With Eventarc, we can create a trigger that listens to these audit logs and sends a HTTP request in the form of a CloudEvent to a Cloud Run service. Here’s an example gcloud command:
So whenever your project sees an audit log with this serviceName and methodName, Eventarc will send a POST request to your Run service with an HTTP body of the audit log.
(Optional) Use a Google CloudEvent Library
Depending on the programming language of your Cloud Run service, you may want to have some type autocompletion for the CloudEvent payload.
Google provides some type libraries that can help your app use these events:
- Node / TypeScript: github.com/googleapis/google-cloudevents-nodejs
- Go: github.com/googleapis/google-cloudevents-go
- Java: github.com/googleapis/google-cloudevents-java
For those curious, these libraries are generated from protobufs and JSON schemas that are open sourced on GitHub here:
Thanks for reading! If you learned something, please smash that 👏!
