Introduction
AWS (Amazon Web Services) has introduced a game-changing feature: the ability to trigger AWS Lambda functions directly from CloudWatch alarms. This new capability streamlines processes, allowing for quick and automated responses to critical events without manual intervention. In this article, we’ll explore the simplicity and efficiency this integration brings to your cloud workflow.
Understanding the feature
Before the introduction of this feature, setting up a response mechanism to CloudWatch alarms involved a multi-step process. When a CloudWatch alarm was triggered due to a predefined condition being met (such as CPU utilisation exceeding a certain threshold), it would typically notify an AWS SNS (Simple Notification Service) topic.
Subsequently, an AWS Lambda function would need to subscribe to this SNS topic to receive notifications. The Lambda function would then execute the desired actions or logic in response to the alarm.
However, with the new capability introduced by AWS, CloudWatch alarms can now directly trigger AWS Lambda functions without the need for an intermediate SNS topic. This direct integration streamlines the process significantly, eliminating the additional step of routing through SNS topics.
Now, when a CloudWatch alarm is triggered, it can immediately invoke a specified Lambda function, allowing for a more efficient and simplified workflow. This direct triggering mechanism reduces complexity, latency, and potential points of failure in the event-driven architecture, making it easier for developers to set up and manage their alarm-based automation.
Implementation
- Configuring CloudWatch Alarm: To set up CloudWatch alarms, go to the CloudWatch service in the AWS Console. Create a new alarm, choose what you want to monitor, like CPU usage. Set thresholds for when you want the alarm to go off, like high CPU levels. Next, configure actions to be taken when the alarm state changes, in this case, under Lambda action, select the AWS lambda you want to trigger (You can trigger more than one lambda!!). Review everything and create the alarm.
- Creating Lambda Function: To create Lambda functions, navigate to AWS Lambda in the Console, click “Create function,” choose your runtime, write your code, set permissions, test the integration, and consider automation and scaling options for efficient management.
- Add permissions in Lambda: In the AWS Lambda Console, go to your function’s configuration.
- Under the Permissions section, navigate to Resource-based policy statements.
- Select Add permissions and choose AWS service among options.
- In the dropdown list, select Other.
- Add a unique statement ID.
- For Principal, add “lambda.alarms.cloudwatch.amazonaws.com.”
- Under Source ARN(Amazon Resource Names), add the CloudWatch alarm ARN.
- In the Action dropdown list, select “lambda:InvokeFunction.”
- Click Save to apply the configuration.
Official Documentation: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html#alarms-and-actions (Please note that the json policy outlined in the documentation might not be correct and there could be updates or corrections since the time of writing this article.)
Confirmation of a successful trigger:
- AWS console > Cloudwatch service > Alarms > Your alarm.
- Within the alarm details page, click on the History tab to view the list of events related to the alarm. One such event would be of Type: Action and Summary: Successfully executed action arn:aws:lambda:eu-west-1:123456789012:function:test-lambda-function
Sample Event object to AWS Lambda:
{
"source": "aws.cloudwatch",
"alarmArn": "arn:aws:cloudwatch:eu-west-1:123456789012:alarm:test-dlq-alarm",
"accountId": "123456789012",
"time": "2024-02-11T18:35:58.664+0000",
"region": "eu-west-1",
"alarmData": {
"alarmName": "test-dlq-alarm",
"state": {
"value": "ALARM",
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [7.0 (11/02/24 18:34:00)] was greater than or equal to the threshold (4.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2024-02-11T18:35:58.661+0000\",\"startDate\":\"2024-02-11T18:34:00.000+0000\",\"statistic\":\"Minimum\",\"period\":60,\"recentDatapoints\":[7.0],\"threshold\":4.0,\"evaluatedDatapoints\":[{\"timestamp\":\"2024-02-11T18:34:00.000+0000\",\"sampleCount\":1.0,\"value\":7.0}]}"
},
"previousState": {
"value": "OK",
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [0.0 (11/02/24 18:32:00)] was not greater than or equal to the threshold (4.0) (minimum 1 datapoint for ALARM -> OK transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2024-02-11T18:33:58.729+0000\",\"startDate\":\"2024-02-11T18:32:00.000+0000\",\"statistic\":\"Minimum\",\"period\":60,\"recentDatapoints\":[0.0],\"threshold\":4.0,\"evaluatedDatapoints\":[{\"timestamp\":\"2024-02-11T18:32:00.000+0000\",\"sampleCount\":1.0,\"value\":0.0}]}"
},
"configuration": {
"metrics": [
{
"id": "86c6ccb6-f926-dcea-753f-5190e142122b",
"metricStat": {
"metric": {
"namespace": "AWS/SQS",
"name": "ApproximateNumberOfMessagesVisible",
"dimensions": {
"QueueName": "test-dlq"
}
},
"period": 60,
"stat": "Minimum"
},
"returnData": true
}
]
}
}
}Use case
In our team’s journey to streamline DLQ(Dead-letter queue) management across numerous AWS SQS (Simple Queue Service) queues, we initially came up with a solution leveraging CloudWatch alarms, SNS topic, and Lambda functions for redrive operations. This approach was implemented successfully in our QA environment.
However, as we prepared to deploy this solution into production, AWS added this new feature: direct triggering of Lambda functions from CloudWatch alarms. With this direct integration, our DLQ redrive process became efficient and simplified our architecture.
Post Script
- In Dec 2021, AWS released DLQ redrive to the source queue.
- In June 2023, it launched APIs to programatically redrive DLQs.
- In Dec 2023, it finally added AWS Lambda as an alarm state change action. (Apparently AWS enthusiasts have waited for close to a decade for this feature!!)
Thank you AWS for continuous evolution and innovation within your ecosystem.