AWS CloudFormation: Defining Lambda Backed Custom Resources

In my previous post, I’ve mentioned about some practices to improve cost efficiency on cloud solutions. The first section in that story was about being in control of your cloud resources. Best way to achieve this is of course automating your provisioning and deployment processes.

AWS CloudFormation is an AWS service that provides a common language for defining AWS resources as a code. Using AWS CloudFormation, you can define almost any AWS resource type and also add custom business logic using custom resources.

In this story, I will not dive into details of how to define AWS resources with CloudFormation for two reasons. First, there are too many resource types to cover in one story and second, I think the AWS CloudFormation documentation is pretty clear.

This story is about defining custom resources, instead to handle custom business logic during resource provisioning or deployment. I’ll show how to trigger a lambda function from CloudFormation where we can add any custom provisioning step (like inserting or updating some records on the database) and return response back to CloudFormation.

The CloudFormation Template

In this CloudFormation template example there are 3 resource definitions:

• lambda execution role
• lambda function definition
• custom resource definition

Lambda Execution Role

We need the lambda execution role to be able to invoke the lambda function.

"MyLambdaExecutionRole": {
  "Type": "AWS::IAM::Role",
  "Properties": {
    "AssumeRolePolicyDocument": {
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": [
              "lambda.amazonaws.com"
            ]
          },
          "Action": [
            "sts:AssumeRole"
          ]
        }
      ]
    },
    "Policies": [
      {
        "PolicyName": "AllowLogging",
        "PolicyDocument": {
          "Statement": [
            {
              "Effect": "Allow",
              "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
              ],
              "Resource": "arn:aws:logs:*:*:*"
            }
          ]
        }
      }
    ],
    "Description": "A simple role for executing lambda functions with no permissions",
    "RoleName": "MyLambdaExecutionRole",
    "Path": "/my/path"
  }
}

Lambda Function Definition

The lambda function definition given below, points to a local jar file in the “Code” property.

Alternatively, you can upload your artifacts to an S3 bucket manually and point there in your template.

If you choose to continue with pointing to a local artifact, you should process your template with the cloudformation package command (before deploying it), which uploads the local artifacts to an S3 bucket that you specify and replaces the local artifact references in your template with that S3 location.

Either way, the IAM account that will execute the CloudFormation template, needs to have a read privilege to the corresponding S3 bucket.

"MyLambdaFunction": {
  "Type": "AWS::Lambda::Function",
  "Properties": {
    "FunctionName": "MyLambdaFunctionName",
    "Handler": "com.mypackage.CustomResourceHandler::handleRequest",
    "Code": "target/my-artifact-jar-with-dependencies.jar",
    "Runtime": "java8",
    "MemorySize": 128,
    "Role": {
      "Fn::GetAtt": [
        "MyLambdaExecutionRole",
        "Arn"
      ]
    },
    "Timeout": 120,
    "Environment": {
      "Variables": {
        "ActiveProfiles": {
          "Ref": "ActiveProfiles"
        }
      }
    }
  }
}

The Custom Resource That Triggers The Lambda Function

The custom resource definition given below will automatically trigger the lambda function after that lambda function is created.

"MyCustomResource": {
    "Type": "Custom::LambdaInvoker",
    "DependsOn": [
      "MyLambdaFunction"
    ],
    "Properties": {
      "ServiceToken": {
        "Fn::GetAtt": ["MyLambdaFunction","Arn"]
      },
      "Region": {
        "Ref": "AWS::Region"
      },
      "myFirstParameter": {
        "Ref": "Param1"
      },
      "mySecondParameter": {
        "Ref": "Param2"
      }
    }
  }
}

I passed the ARN of the lambda function to the custom resource definition as a parameter (see the ServiceToken property). By doing this, I tell the CloudFormation to invoke that lambda function during the creation, update, or deletion of this custom resource.

You can define any parameter you like in your custom resource definition. Those parameters will be passed to your lambda function within the custom resource request object.

Your lambda function will receive an input in the following format:

{
  "RequestType": "Create",
  "ResponseURL": "http://...",
  "StackId": "stack-1",
  "RequestId": "Request-1",
  "ResourceType": "Custom",
  "LogicalResourceId": "Logical-Resource-1",
  "ResourceProperties": {
    "myFirstParameter": "123",
    "mySecondParameter": "asd",
  }
}

The RequestType can be either Create, Update or Delete. If you are creating a new CloudFormation stack, it will be Create. If you are updating or deleting the stack that you’ve previously created, then it will be Update or Delete correspondingly.

One thing to pay attention is the ResponseUrl. While creating, updating or deleting a custom resource, the CloudFormation will wait until you return a response to the provided ResponseURL in the request object. And you’ll not be able to delete the stack during that time period until you get timeout (which is 1 hour).

The ResourceProperties is the final field that I want to mention. ResourceProperties is a JSON object that contains the parameters you pass from your custom resource definition in your CloudFormation template.

See the Custom Resource Request Fields and the Custom Resource Response Fields for more information.

Getting the Custom Resource Request Object and Returning Response to Given Response URL

I’ve used the cfn-response library in my lambda function which includes Custom Resource Request and Response model as well as implementation for sending response back to the pre-signed S3 URL (the ResponseURL).

<!-- Maven Dependency -->
<dependency>
    <groupId>com.sunrun</groupId>
    <artifactId>cfn-response</artifactId>
    <version>${cfn-response.version}</version>
</dependency>
// The handler function
public class CustomResourceHandler {

    public void handleRequest(final CfnRequest<MyDataType> request, Context context) {
            // Your custom operations goes here
            this.sendResponse(request, context, Status.SUCCESS);
        }
    }
}

// Sending Response to ResponseURL
public void sendResponse(CfnRequest<MyDataType> request, Context context, Status status, String reason) {
    CfnResponseSender sender = new CfnResponseSender();
    // generate your physical id
    sender.send(request, status, context, reason, null, physicalId);
}

I hope this story helped you with defining custom resources using AWS CloudFormation.