Configure CAI Feed for Real Time Notifications about GCP Resource Changes

GCP CAI Feed is a feature of Cloud Asset Inventory that allows you to receive real-time notifications about changes to your Google Cloud resources and policies. The following steps shows how we can configure it in GCP.

Create the Pub/Sub topic

Create a Pub/Sub topic where a message is sent when an asset is changed.

$ PROJECT_ID=my-project-id
$ CAI_FEED_TOPIC_ID=cai_feed
$ gcloud pubsub topics create $CAI_FEED_TOPIC_ID --project $PROJECT_ID

Add permission for CAI agent SA to the pub/sub topic

Assign permission to Cloud Asset Inventory agent to publish message to the topic created above.

$ gcloud pubsub topics add-iam-policy-binding \
projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--member=serviceAccount:service-$PROJECT_ID@gcp-sa-cloudasset.iam.gserviceaccount.com \
--role=roles/pubsub.publisher

[Optional] Create the subscription to the Cloud Run API endpoint

Create a Pub/Sub subscription of type Push that is used to receive any message whenever an asset is changed. Here, Cloud Run will be used as an example, you can define your own subscription as well.

$ CLOUDRUN_API_URL="https://my-cloud-run-url/path"

# SERVICE_ACCOUNT_EMAIL is the email address of the service account granted cloud run invoker role
# Ref: https://cloud.google.com/run/docs/authenticating/service-to-service#set-up-sa
$ SERVICE_ACCOUNT_EMAIL="SERVICE_ACCOUNT_EMAIL@myorg.com"
$ gcloud pubsub subscriptions create cai_feed_subscription \
--topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--push-endpoint="$CLOUDRUN_API_URL" \
--push-auth-service-account=${SERVICE_ACCOUNT_EMAIL} \
--ack-deadline=600 \
--min-retry-delay=30s \
--max-retry-delay=600s

Create the feed

A CAI feed has to be created to ensure Pub/Sub messages are sent to the topic created above. Depending on the requirements, the CAI feed have to be created:

• At the organization level, if the customer would like to receive any asset changes for a full organization
• At the folder level, if the customer would like to receive any asset changes for a specific folder
• At the project level if the customer would like to receive any asset changes for a specific project

“gcloud asset feeds create” is used to create the different feeds. For some parameter explanation:

• -asset-types is a regexp representing the asset types that are part of the CAI feed. Depending of the different need, this parameter can be restricted or extended to ensure the CAI feed is only working for the intended asset type, for full list of supported assets can be referred here.
• –content-type represents the asset content type part of the feed. It can be one of those values: resource, iam-policy and org-policy

Organization Feed

$ ORGANIZATION_ID=<YOUR_ORG_ID>
$ CAI_FEED=cai_feed

$ gcloud asset feeds create ${CAI_FEED}_resource \
--organization=$ORGANIZATION_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=resource

$ gcloud asset feeds create ${CAI_FEED}_iam-policy \
--organization=$ORGANIZATION_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=iam-policy

$ gcloud asset feeds create ${CAI_FEED}_org-policy \
--organization=$ORGANIZATION_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=org-policy

Folder Feed

$ REALTIME_FOLDER_ID=<YOUR_REALTIME_FOLDER_ID>
$ CAI_FEED=cai_feed

$ gcloud asset feeds create ${CAI_FEED}_resource \
--folder=$REALTIME_FOLDER_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=resource

$ gcloud asset feeds create ${CAI_FEED}_iam-policy \
--folder=$REALTIME_FOLDER_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=iam-policy

$ gcloud asset feeds create ${CAI_FEED}_org-policy \
--folder=$REALTIME_FOLDER_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=org-policy

Project Feed

$ REALTIME_PROJECT_ID=<YOUR_REALTIME_PROJECT_ID>
$ CAI_FEED=cai_feed

$ gcloud asset feeds create ${CAI_FEED}_resource \
--project=$REALTIME_PROJECT_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=resource

$ gcloud asset feeds create ${CAI_FEED}_iam-policy \
--project=$REALTIME_PROJECT_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=iam-policy

$ gcloud asset feeds create ${CAI_FEED}_org-policy \
--project=$REALTIME_PROJECT_ID \
--pubsub-topic=projects/$PROJECT_ID/topics/$CAI_FEED_TOPIC_ID \
--asset-types="^.*.googleapis.com/.*$" \
--content-type=org-policy

Reference

Monitoring asset changes | Cloud Asset Inventory Documentation | Google Cloud