Trigger a Cloud Run service starting from an (Apigee) audit log event with EventArc

TL;DR

This article provides a step-by-step guide on how to build the necessary architecture to trigger a Cloud Run service from an Audit Log Event with Eventarc. For the context of this article, we’ll leverage Apigee Audit Logs but you can take advantage of the same architecture for any of the events supported by Eventarc.

Context

When you start a conversation about Cloud Operations with almost any customer, one of the first requirements that pops up is “How can I monitor who-does-what within my GCP Organisation?”. Right after you mention the magic term audit logs, the follow up question usually is “… and how can I react following a specific audit log event generated by a user action?”

This is exactly what we want to address in this article. When I say “we”, it’s because I had the chance (and the great opportunity) to collaborate and co-author this article with two colleagues, Miren Esnaola and Anmol Krishan Sachdeva (a huge thanks to them).

Going back to the problem that we are trying to solve, our initial idea was to take advantage of the “de-facto” solution composed by:

• Cloud Logging: to retrieve and forward just the audit log events we were looking for through log filter and log sink
• Pub/Sub: as the event-driven system receiving the filtered log events from Cloud Logging and forwarding them to Cloud Run
• Cloud Run: receiver of the messages subsequently delivered from PubSub as “push” HTTP requests and performing a given (whatever is the customer requirement) action

This solution (described in detail here) can be used as Swiss knife to cover a tons of use-cases, but starting from a customer’s requirements where we need to capture and react specifically to Apigee audit events, we were wondering if we could leverage even a more cloud-native solution and we came across Eventarc.

Eventarc

From the official documentation

Eventarc lets you asynchronously deliver events from Google services, SaaS, and your own apps using loosely coupled services that react to state changes. Eventarc requires no infrastructure management — you can optimize productivity and costs while building a modern, event-driven solution

By reading this definition and applying to our use-case:

• An Audit Log is a type of “event generated by a Google Service” (Apigee in this case)
• The “requires no infrastructure management” feature was exactly what we were looking for, an even more cloud-native solution than the one previously mentioned

Eventarc Architecture

Indeed, Eventarc gives you the possibility to create event-driven workflows based on audit log events, for example, you could create a trigger that sends an email notification whenever a new user is created in your Google Cloud Project or any other automated remediation process you’d like to set up.

Cloud Audit Logs Events is just one of the three supported event types by Eventarc, the others are “Direct Events” and “Third-Party Events”. Below is a brief recap:

• Cloud Audit Logs (CAL) Events: this is what we are looking for. In this case, the Eventarc triggers with type=google.cloud.audit.log.v1.written send requests to your service(s) (in our case Cloud Run) when an audit log is created that matches the trigger’s filter criteria.
• Direct Events: on the other side, natively, Eventarc can also be triggered by various direct events, each of them having a specific type as listed here and the list of out-of-the-box cloud events keeps growing at a rapid pace with new developments happening in Google products. This should be the preferred method when possible.
• Third-Party Events: these are the events from third-party sources like Check Point CloudGuard, Datadog, ForgeRock, Lacework, etc. It should be noted that support for these third-party sources is currently in preview.

Let’s put all the pieces together: how do I trigger my Cloud Run service following an Apigee event?

Now that we have found the tool to accomplish our goal, just as simple as that, the following graph shows what your architecture will look like if you take advantage of Eventarc to trigger Cloud Run starting from a “delete API Proxy” event.

End-to-end architecture

The end-to-end flow of actions is quite simple:

1. A user or an application deletes an API Proxy either through UI or API
2. Apigee generates a Cloud Audit logs event logging what happened
3. Eventarc is constantly pulling these logs from Cloud Logging and when it finds one, triggers the desired Cloud Run service

Step-by-step guide

From the implementation point of view, the following list of steps allows you to recreate the solution we just designed but it assumes you have your own Apigee deployment set up and ready to be used.

For our tests, we have used a simple Cloud Run service (called “helloworld-events”) logging whatever it receives to Cloud Logging. You can find the open-source code here.

1. First, let’s set some environment variables to be used during our guide

export PROJECT_ID=test-eventark-with-apigee
export SERV_ACCOUNT=eventark-to-run

2. Second step, enable the necessary API in your project

gcloud services enable logging.googleapis.com \
 eventarc.googleapis.com \
 eventarcpublishing.googleapis.com \
 run.googleapis.com \
 pubsub.googleapis.com

3. Create the user-managed service account to allow Eventarc to publish on Cloud Run

gcloud iam service-accounts create $SERV_ACCOUNT
gcloud projects add-iam-policy-binding $PROJECT_ID \
 --member=serviceAccount:eventark-to-run@${PROJECT_ID}.iam.gserviceaccount.com \
 --role=roles/eventarc.eventReceiver
gcloud projects add-iam-policy-binding veronica-test-demo \
 --member=serviceAccount:eventark-to-run@${PROJECT_ID}.iam.gserviceaccount.com \
 --role=roles/run.invoker
gcloud projects add-iam-policy-binding $PROJECT_ID \
 --member=serviceAccount:eventark-to-run@${PROJECT_ID}.iam.gserviceaccount.com \
 --role=roles/iam.serviceAccountTokenCreator

4. Last but not least, let’s create the necessary trigger for our use case by specifying:

event-filter-type: google.cloud.audit.log.v1.written
event-filter-service-name: apigee.googleapis.com
event-filter-method-name: google.cloud.apigee.v1.ApiProxyService.DeleteApiProxy

gcloud eventarc triggers create apigee-to-run-trigger \
 --location=global \
 --destination-run-service=helloworld-events \
 --destination-run-region=europe-west2 \
 --event-filters="type=google.cloud.audit.log.v1.written" \
 --event-filters="serviceName=apigee.googleapis.com" \
 --event-filters="methodName=google.cloud.apigee.v1.ApiProxyService.DeleteApiProxy" \
 --event-data-content-type="application/json" \
 --service-account=${SERV_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com

My Eventarc trigger detail

One important requirement to keep in mind is that Eventarc catches events just from the project where is deployed.

Other Apigee use cases you might want to try

Here few more use cases you might want to try

1. Capture Apigee Developer creation with google.cloud.apigee.v1.Developers.CreateDeveloper and take some action; credits to Dino Chiesa and Ganapathy Duraiswamy
2. Automatic approval for the new Apigee Apps that require manual approval
3. Automatic addition, modification, removal, and publishing of APIs in Apigee Developer Portal and/or API Hub
4. Track status of Apigee Runtime Instances with google.cloud.apigee.v1.RuntimeService.ReportInstanceStatus and take further actions
5. Send notifications when Develop App Keys are replaced or deleted
6. API Hub integration with Apigee using Eventarc for CI/CD Automation; credits to Nicola Cardace

Some more ideas can be found along with sample code snippets in the great article written by Daniel Strebel.

Conclusion

The Apigee-Eventarc integration is a really powerful tool you can add add to your SIEM overall architecture to make it even more cloud-native, in particular if your automated remediation process can be easily implemented in Cloud Run.