A universal lambda function to monitor the health of multiple applications and trigger SNS alerts
Problem
We are running a bunch of Spring Boot applications in the AWS cloud. We have enabled a spring actuator that exposes a health check endpoint ( actuator\health )
Even though the ELB health checks monitor these and respawn the errored out applications, we have seen situations where even the newly spawned app instances continue to crash and introduce downtimes.
So we needed a way to get notified if an application is down so we can manually intervene.
Solution
The above diagram explains the serverless monitoring architecture I came up with to solve this problem.
- A lambda function written in Python that can check a set of health check endpoints when invoked (Code is shown in a later section)
- A schedule defined in Cloud watch with the above lambda as a target. In our case, the schedule was every 5min.
- A SNS topic configured to send text messages and emails
How does this work?
The lambda uses Python 3.9 runtime. This lambda function should have an IAM role that allows publishing to SNS.
Below is the code we are running in the lambda function. It is very lean and uses no other dependencies.
The code is very easy to understand. You can put the list of health check endpoints in the URLs array at the top.
Once this runs, if one or more URLs returns a none 200 response or any other error it will send that information to the SNS topic and then the SNS topic will publish the messages to the subscribers.
To invoke the Lambda function every 5min, we configured a cloud watch scheduler with the lambda function as the target. For more information please follow this. https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html