Serverless — Cloud Custodian setup on AWS Lambda

Cloud Custodian is CapitalOne’s open source product for managing your cloud. It provides various functionalities such as:

  1. Cost Management
  2. Compliance

You can read more about this tool at CapitalOne’s developer website

This blog will explain how to setup a simple tag-compliance policy using cloud custodian on AWS lambda making it completely serverless. This policy will check for specific tag on EC2 instances and if absent, it will perform the defined action , in this case STOP the instance.

Prerequisites:

  1. AWS Account
  2. IAM access on the AWS account to create policies, roles and user
  3. If using Windows, you need either virtual machine or docker setup on your machine where linux VM or container can be run. If using linux/unix, nothing to do.

IAM Role:

Lambda function requires access to perform action on EC2 instances. This can be achieved by attaching an IAM role to this lambda function. If you don’t know how to create an IAM role, read How to create IAM role .

Create and IAM role “custodian-tag-compliance-role” and grant AmazonEC2FullAccess policy to this role.

IAM User:

To create lambda function from your local machine, you need programmatic access on AWS. For this, create and IAM user “custodian-bot” and grant the following permissions/policies:

  1. AWSLambdaFullAccess (AWS managed policy)
  2. CloudWatchFullAccess (AWS managed policy)
  3. IAMPassRole
{
“Version”: “2012–10–17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: “iam:PassRole”,
“Resource”: “arn:aws:iam::1234567890:role/lambda_basic_execution”
}]
}

Create programmatic access for this user. This will provide you with access key and secret key.

Copy the ARN of this IAM user and add in the trust relationship of the IAM role created before(custodian-tag-compliance-role)

Install Cloud Custodian:

If you are using Windows OS on your local machine, you need to have either one of the following:

  1. Virtual Machine installed to run linux
  2. Docker installed to run linux container

Once you have either of the above, switch to the linux environment and install cloud custodian. In order to install and execute cloud custodian, you need to have python and pip installed. If you do not have either of them, install and then proceed further.

Set the environment variables with the access key & secret key of the user and the default region.

export AWS_ACCESS_KEY_ID=<ACCESS_KEY>
export AWS_SECRET_ACCESS_KEY=<SECRET_KEY>
export AWS_DEFAULT_REGION=<REGION>

Install cloud custodian using pip

pip install c7n

To check the installation version, use the following command:

custodian version

Once the installation is complete, create the policy document. Cloud custodian policy is a YAML document which defines the policies and actions to be taken on cloud resources. Further reading on how to write a policy can be found at policies.

The policy we will be using is :

policies:
- name: owner-tag-compliance
mode:
type: periodic
schedule: rate(1 hour)
role: arn:aws:iam::1234567890:role/Custodian-tag-compliance-role
resource: ec2
description: |
Schedule a resource that does not meet tag compliance policies
to be stopped in four days.
filters:
- State.Name: running
- "tag:Owner": absent
actions:
- stop

This policy will create a lambda function with role Custodian-tag-compliance-role and a cloudwatch rule which will trigger the lambda function every 1 hour and check EC2 resources for the following filters:

  1. The instance is in running state
  2. The “Owner” tag is absent

Instances matching this criteria will be stopped.

Save the file with the name policy.yml

Run the following command to create the lambda function to execute the policy , the cloudwatch rule for the schedule event.

custodian run -s . policy.yml

This lambda function will run every hour and check for the filters. If intances are found with the filters, it will STOP them. To test this function you can go to the AWS lambda console and trigger the function manually with the default test event.

If you want to see various commands with custodian,

custodian -h

References:

https://developer.capitalone.com/opensource-projects/cloud-custodian/