Using Cloud Asset Inventory feeds for dynamic configuration and policy enforcement

One underappreciated feature of Cloud Asset Inventory are feeds, which monitor resource and policy changes in real-time via a stream of events published to PubSub queues.

Wired to a Cloud Function, they expose the full set of resource attributes in each message and allow simple resource tracking, dynamic configuration, or policy enforcement with a minimal amount of code, and can be leveraged to address some common use cases:

• updating a remote data source on resource changes, e.g. an enterprise CMDB
• managing dependent resources, e.g. custom Cloud DNS records for instances and forwarding rules
• triggering alerts on changes to critical resources or IAM bindings
• enforcing a predefined policy on the configuration of specific resources, e.g. a Cloud Armor policy on backend services

The code shown here falls into the last category, enforcing a predefined policy on VM network tags, by detecting and reverting non-compliant changes in near-real time. It’s a flexible and simple way to add an additional measure of control when using tags to scope firewall rules.

The example is designed to be minimal and self-contained, in actual use some changes would of course be needed: scoping the feed and IAM custom role/bindings for the Cloud Function to the organization or a folder; tuning the function’s retry and error handling; and putting in place a separate way to periodically run blanket wide checks (possibly using Cloud Asset exports).

This is a high-level overview of the resources that will be created by the example code:

To run the example, start by cloning the repository and changing to the example folder. If you prefer using Cloud Shell, this link will run the Git clone for you, and switch to the correct folder so that you can skip the initial step below.

git clone https://github.com/terraform-google-modules/cloud-foundation-fabric.git
cd blueprints/cloud-operations/asset-inventory-feed-remediation

Once in the right folder, run Terraform and just provide the project id where the required resources will be created (we will clean up later, don’t worry). The $GOOGLE_PROJECT_ID variable in the command below is set by Cloud Shell to the current project in the Cloud console, if you’re not in Cloud Shell or want to use a different project, replace it with the actual project id.

terraform init
# answer 'yes' when prompted after running apply
terraform apply -var project_id=$GOOGLE_PROJECT_ID

When Terraform has finished running it will print a few gcloud commands needed for testing, and the command to create the feed which is not yet supported in the Google Terraform provider (run terraform output to print the commands on screen again if you scroll past them).

Copy and run the command in the feed_create output, which should be similar to this (don’t copy and paste from here!):

gcloud asset feeds create feeds-test \
  --pubsub-topic projects/feeds-test/topics/feeds-test \
  --asset-types compute.googleapis.com/Instance \
  --content-type resource \
  --project feeds-test

We’re now ready to test the example.

First of all, here is how the tag policy is implemented in the Cloud Function:

• each time an instance is created or changed (status change is not RUNNING or TERMINATED) the Cloud Function checks its tags
• it discards any tag that does not start with a preset prefix (shared-or gke-cluster-) or with the project id
• then compares the resulting filtered tags with those on the instance, if it finds a difference it calls the setTags Compute Engine API method to remove the non-compliant tags

To test the function add a non-compliant tag to the instance, either via the console or using the tag_add command displayed in the Terraform output. Then check the Cloud Function logs, again via the console or the cf_logs command in the Terraform output.

You should see two separate function invocations: the first change adding the non-compliant tag, and the second change done by the function removing it. These are actual log entries from the same function, in reverse order (the gcloud logging read command defaults to descending sort) and slightly edited for clarity:

DEBUG Function execution started
INFO  parsing event data
INFO  checking https://www.googleapis.com/.../feeds-test-1
INFO  tags ['foobar', 'shared-test-feed', 'xxxxx-test-feed']
INFO  modify tags xxxxx europe-west1-b feeds-test-1 XrN1ewa5m98= ['shared-test-feed', 'tf-playground-simple-test-feed']
DEBUG Function execution took 6153 ms, finished with status: 'ok'
DEBUG Function execution started
INFO  parsing event data
INFO  checking https://www.googleapis.com/.../feeds-test-1
INFO  tags ['shared-test-feed', 'xxxxx-test-feed']
INFO  all tags are valid
DEBUG Function execution took 14 ms, finished with status: 'ok'

You can confirm that the change performed by the function was successful by looking at the instance details with the tag show command:

tags:
  fingerprint: HwowTSTO_e8=
  items:
  - shared-test-feed
  - xxxxx-test-feed

When you’re done testing, simply run terraform destroy to delete all create resources, then delete the feed by hand using gcloud asset feeds delete [feed name] --project [project id].

I hope this short example was enough to show the power of Cloud Asset feeds, and to give you a head start on using them.