Saving money by automatically shutting down RDS instances using AWS Lambda and AWS SAM
A complete walk-through with code snippets and full code repo
At Hatch we rigorously follow up the cloud operation costs for all our clients. We do this for live systems, but also during the development and testing phase.
While production databases need to be up and running 24/7, this usually isn’t the case for databases in a development or test environment. By shutting them down outside business hours, our clients save up to 64%.
E.g. for only one db.m5.xlarge RDS instance in a single availability zone this already means a saving of $177/month or $2,125/year.
In the next minutes, I’ll walk you through the different steps of creating Lambda functions that will automatically shut down and start up your database by using AWS SAM.
- Install Node.js
- Install AWS CLI and configure it
- Install AWS SAM
- Install Docker (only needed for testing locally)
To test our code we will use the AWS CLI to setup a MySQL database on a db.t2.micro instance.
aws rds create-db-instance --db-instance-identifier testdb --db-instance-class db.t2.micro --engine mysql --allocated-storage 20 --master-username admin --master-user-password adminPwd
AWS SAM will need a S3 bucket to store its deployment artifacts. We create one using the command below.
aws s3 mb s3://auto-aws-db-shutdown-deployment-artifacts
The basic project setup is extremely simple. We only need one file, i.e. template.yaml.
We don’t need a package.json file for installing additional dependencies because our function code will only use the AWS SDK for Node.js and AWS Lambda already includes this as part of the execution environment.
Create Lambda functions
To structure our code we will create two folders, one for the shutdown function and one for the startup function.
In each of the folders we add two files.
- app.js: this file will contain the handler function of our Lambda
- stop.js / start.js: the file that will contain the respective shutdown or startup logic
In general it is a good practice to always separate the handler function and actual logic. This makes the logic easier to unit test.
Let’s have a look at the code that makes it all happen.
Let’s update our template.yaml file and have a look at its structure.
In this section we can define parameters that can be used throughout the rest of the template. You typically create parameters to customize your templates (e.g. per environment). Parameters enable you to specify custom values at deployment time.
In our case we add a parameter for the database InstanceID that defaults to the ‘testdb’ we created earlier.
This section contains properties that are common to all the functions that are listed in the Resources section. Both our functions will have the same runtime, handler name, memory and timeout settings. They also reference the same InstanceID parameter as an environment variable that will be used inside the handler code.
Here we define the actual shutdown and startup functions. You can see the only property that is specific to these functions is the CodeUri. This is the path to the folder where the handler code resides.
Now that we have our code and template file, we can start testing. We do this by using one of the AWS SAM features I like the most, i.e. the possibility to invoke and run Lambda functions on your local machine. If you want, you can even debug your code using one of the AWS Toolkits.
To invoke the shutdown Lambda function locally, we run the command below.
sam local invoke --no-event DBShutDownFunction
When we take a look at our RDS databases in the AWS Management Console we see the database is stopping.
Once the database is stopped, let’s see if we can get it back to life by running the command below.
sam local invoke --no-event DBStartUpFunction
As expected, we see that the database is starting and becomes available after a while.
Deploy Lambda functions
We are now sure our code works, so let’s package our Lambda functions and deploy them to AWS.
sam package --template-file template.yaml --s3-bucket auto-aws-db-shutdown-deployment-artifacts --output-template-file outputTemplate.yaml
This command will upload the artifacts to the S3 bucket we specified and output a new template file called outputTemplate.yaml. We will use this new template file in the next step.
sam deploy --template-file outputTemplate.yaml --stack-name auto-db-shutdown-stack --capabilities CAPABILITY_IAM
This command will create a changeset and execute it. The execution in our case will create a new CloudFormation stack.
Now that our code is deployed, we should find our Lambda functions in the AWS Management Console.
Test on AWS
In order to test the shutdown function, we simply create an empty event and hit the ‘Test’ button.
This time we aren’t lucky. Our test fails because of an AccessDenied error.
Weird? No, not really.
Locally the Lambda function was using the AWS credentials of my personal account with elevated permissions, therefor we could stop and start the database instance. Our Lambda function does not have the proper permissions to do this, so we will need to add a policy.
Luckily, assigning a policy can easily be done inline in the AWS SAM template. We add one policy to allow our shutdown function to stop our database instance and one policy to allow our startup function to start our database instance.
When we package and deploy, we can test our shutdown function again and we see that this time it does work.
Schedule shut down / start up
Our goal was to save some money, so we now need to make sure the database goes down when it won’t be used. We will shut it down between 7 PM and 7 AM on weekdays and completely in the weekend.
We do this by adding scheduled CloudWatch events as triggers for our functions. Don’t forget the cron timings need to be specified in UTC.
When we package and deploy, we see the correct CloudWatch events are added to our Lambda functions.
We can easily monitor the execution of our Lambda functions using CloudWatch, but we don’t want to go and check every day if the shutdown and/or startup was successful.
In fact, we only want to be bothered in case our functions fail. To do so, we will set up an AWS SNS topic that will act as a dead letter queue and link it to our Lambda functions.
In the AWS SAM template, we create the SNS topic with email subscription and reference it as the dead letter queue of our functions.
When we package and deploy again, we can see that the SNS topic was created.
We also receive an email prompting us to confirm our subscription to the newly created SNS topic. Once subscribed, we will receive an email in case one of our functions fails.
A word of advice, always clean up any resources you aren’t using anymore. We are trying to save some money here, remember?
In order to delete the CloudFormation stack and all the resources it created, just run the command below.
aws cloudformation delete-stack --stack-name auto-db-shutdown-stack
Also don’t forget to delete the test database
aws rds delete-db-instance --db-instance-identifier testdb --skip-final-snapshot
and the S3 bucket with the deployment artifacts.
aws s3 rb s3://auto-aws-db-shutdown-deployment-artifacts --force
The final code of this project can be found on Github. You can use it directly by just modifying the database instance, ARN and shutdown / startup timings.
You can also use it as a basis to build similar functionality (e.g. automatic shutdown of EC2 instances) or to experiment a bit more with AWS SAM yourself.
In a next blog I’ll explain how to setup CI/CD for this project using AWS CodePipeline and other AWS Developer Tools.
Thanks for reading!