Cloud Custodian for Compliance As Code and Auto-Remediation on Google Cloud Platform (GCP)

Maintaining compliance with industry standards and regulations is crucial to protecting sensitive data and avoiding costly breaches. However, traditional compliance monitoring and remediation methods are often time-consuming and error-prone. Compliance as Code and Auto-Remediation offer an automated solution to this problem, enabling businesses to proactively address security issues and ensure continuous compliance.

I decided to write a series of articles on how can we use Cloud Custodian to implement your own Compliance as Code and Auto-Remediation strategy on Google Cloud , but first lets start with the fundamental.

What is Cloud Custodian ?

Cloud Custodian is an open-source tool designed to help businesses maintain and manage their cloud infrastructure in a compliant and secure manner. It offers a comprehensive set of policies that enable to automate compliance checks, resource management, and security enforcement across their cloud accounts. Cloud Custodian supports various cloud providers, including Google Cloud, Amazon Web Services, and Microsoft Azure.

Cloud Custodian policies are simple yaml file which includes Resource, Filter and Action

• Resources: Custodian is able to target a specific cloud that has its own resource type.(eg ec2, s3 bucket)
• Filters: Filters are the way in Custodian to target a specific subset of resources. It could be based on some date, tag etc. We can write our custom filter using the JMESPath expression.
• Actions: Actions is the actual decision you make on resources that match the filter. This action can be as simple as sending a report to the owner, stating that the resource does not match the Cloud governance rule or delete the resource.

Both actions and filters can combine as many rules as you want to express your needs perfectly.

- name: some-policy
  resource: name-of-cloud-resource
  description: Description of policy
    filters:
      - (some filter that will select a subset of resource)
      - (more filters)
    actions:
      - (an action to trigger on filtered resource)
      - (more actions)

Cloud Custodian and Cloud Functions Integration ?

When integrated with Google Cloud Functions, Cloud Custodian enables automatic remediation in response to policy violations or specific events by defining and enforcing custom policies for managing cloud environments. Users create policies specifying the desired state of resources and the actions to take in case of non-compliance, which are then deployed as Google Cloud Functions.

Upon the occurrence of audit logs event, the Google Cloud Function is triggered and evaluates the state of the affected resources and determines if they violate the specified rules. If a violation is detected, the policy carries out the defined remediation action(s), which may include modifying resource configurations or sending notifications.

Setup Cloud Custodian

Step 1: Prerequisites

• A Google Cloud Platform (GCP) account with appropriate permissions and Cloud Functions API enabled
• Python 3.6 or higher installed on your machine.
• Google Cloud SDK installed and configured on your machine.

Step 2: Installing Cloud Custodian

To install Cloud Custodian and the the GCP-specific extensions, you can use pip, the Python package installer. Open a terminal or command prompt and run the following command:

python3 -m venv custodian
source custodian/bin/activate
pip install c7n_gcp

Step 3: Authenticating Cloud Custodian with GCP

Before using Cloud Custodian, you need to authenticate it with your GCP account. You can do this by setting the GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT environment variables to point to your GCP service account JSON key file and project ID.

export GOOGLE_APPLICATION_CREDENTIALS="< /path/to/your-service-account-key.json >"
export GOOGLE_CLOUD_PROJECT="< google cloud project id>"

Writing your first policy

Create a YAML file called uniform_access_storage_remediation.yml with the following content:

policies:
  - name: gcp-uniform-access-storage-remediation
    resource: gcp.bucket
    filters:
      - type: value
        key: iamConfiguration.uniformBucketLevelAccess.enabled
        value: false
    actions:
      - type: set-uniform-access

This policy checks for GCP storage buckets that have uniform access disabled and and enable it.

Validate and enforce rules

First you need to create a Google Cloud Storage bucket with uniform access control disabled, for example, to create a bucket named “my-example-bucket” ( -b off to disable uniform access):

gsutil mb -p $GOOGLE_CLOUD_PROJECT -c STANDARD -l us-east1 -b off gs://my-example-bucket/

Then you need run the policy, execute the following command in your terminal:

custodian run --cache-period 0 -s output uniform_access_storage_remediation.yml

The output must be something like this :

custodian run -s output uniform_access_storage_remediation.yml
2023-03-30 19:00:19,368: custodian.policy:INFO policy:gcp-uniform-access-storage-remediation resource:gcp.bucket region: count:1 time:0.82
2023-03-30 19:00:19,574: custodian.policy:INFO policy:gcp-uniform-access-storage-remediation action:bucketlevelaccess resources:1 execution_time:0.20

You can see in the output that cloud Custodian did find one bucked with the requested filter and it did the remediation. You can check this in the console.

Setup an automatic remediation using Cloud Functions

To change the policy for automatic remediation using Google Cloud Functions, you’ll need to modify the mode section of the policy. Here’s the updated uniform_access_storage_remediation.yml file:

policies:
  - name: gcp-uniform-access-storage-remediation
    resource: gcp.bucket
    mode:
      type: gcp-audit
      methods:
        - storage.buckets.create
        - storage.buckets.update
    filters:
      - type: value
        key: iamConfiguration.uniformBucketLevelAccess.enabled
        value: false
    actions:
      - type: set-uniform-access

In this policy, we added a mode we set it to gcp-audit. The gcp-audit mode listens for specific Google Cloud audit logs events, such as storage.buckets.create and storage.buckets.update. The policy will be triggered when any of these events occur.

Now, use the same custodian command to deploy the policy as a Google Cloud Function:

custodian run --cache-period 0 -s output uniform_access_storage_remediation.yml

This command will create a new Google Cloud Function with the specified configuration and policy.

With this updated policy and deployment, the automatic remediation of non-uniform cloud storage buckets will be executed in response to Google Cloud audit log events, ensuring a more proactive approach to enforcing security policies.

You can re-run the same test by creation a bucket with uniform access disabled and check that everything is working file.

Conclusion

Cloud Custodian provides a robust framework for setting up an auto-remediation system on Google Cloud Platform (GCP). By leveraging its comprehensive policy language and extensible architecture, organizations can effectively monitor, enforce, and remediate their cloud environments in real-time.

Originally published at https://hassene.belgacem.io .