Saving money by automatically shutting down RDS instances using AWS Lambda and AWS SAM

A complete walk-through with code snippets and full code repo

Maarten Thoelen
Oct 15, 2019 · 7 min read

At Hatch we rigorously follow up the cloud operation costs for all our clients. We do this for live systems, but also during the development and testing phase.

While production databases need to be up and running 24/7, this usually isn’t the case for databases in a development or test environment. By shutting them down outside business hours, our clients save up to 64%.
E.g. for only one db.m5.xlarge RDS instance in a single availability zone this already means a saving of $177/month or $2,125/year.

In the next minutes, I’ll walk you through the different steps of creating Lambda functions that will automatically shut down and start up your database by using AWS SAM.

Prerequisites

Setup

To test our code we will use the AWS CLI to setup a MySQL database on a db.t2.micro instance.

AWS SAM will need a S3 bucket to store its deployment artifacts. We create one using the command below.

The basic project setup is extremely simple. We only need one file, i.e. template.yaml.

We don’t need a package.json file for installing additional dependencies because our function code will only use the AWS SDK for Node.js and AWS Lambda already includes this as part of the execution environment.

Create Lambda functions

To structure our code we will create two folders, one for the shutdown function and one for the startup function.

In each of the folders we add two files.

  • app.js: this file will contain the handler function of our Lambda
  • stop.js / start.js: the file that will contain the respective shutdown or startup logic

In general it is a good practice to always separate the handler function and actual logic. This makes the logic easier to unit test.

Let’s have a look at the code that makes it all happen.

shut-down/app.js
shut-down/stop.js
start-up/app.js
start-up/start.js

Template file

Let’s update our template.yaml file and have a look at its structure.

In this section we can define parameters that can be used throughout the rest of the template. You typically create parameters to customize your templates (e.g. per environment). Parameters enable you to specify custom values at deployment time.

In our case we add a parameter for the database InstanceID that defaults to the ‘testdb’ we created earlier.

This section contains properties that are common to all the functions that are listed in the Resources section. Both our functions will have the same runtime, handler name, memory and timeout settings. They also reference the same InstanceID parameter as an environment variable that will be used inside the handler code.

Here we define the actual shutdown and startup functions. You can see the only property that is specific to these functions is the CodeUri. This is the path to the folder where the handler code resides.

Test locally

Now that we have our code and template file, we can start testing. We do this by using one of the AWS SAM features I like the most, i.e. the possibility to invoke and run Lambda functions on your local machine. If you want, you can even debug your code using one of the AWS Toolkits.

To invoke the shutdown Lambda function locally, we run the command below.

When we take a look at our RDS databases in the AWS Management Console we see the database is stopping.

Once the database is stopped, let’s see if we can get it back to life by running the command below.

As expected, we see that the database is starting and becomes available after a while.

Deploy Lambda functions

We are now sure our code works, so let’s package our Lambda functions and deploy them to AWS.

This command will upload the artifacts to the S3 bucket we specified and output a new template file called outputTemplate.yaml. We will use this new template file in the next step.

This command will create a changeset and execute it. The execution in our case will create a new CloudFormation stack.

Now that our code is deployed, we should find our Lambda functions in the AWS Management Console.

Test on AWS

In order to test the shutdown function, we simply create an empty event and hit the ‘Test’ button.

This time we aren’t lucky. Our test fails because of an AccessDenied error.

Weird? No, not really.
Locally the Lambda function was using the AWS credentials of my personal account with elevated permissions, therefor we could stop and start the database instance. Our Lambda function does not have the proper permissions to do this, so we will need to add a policy.

Assign policy

Luckily, assigning a policy can easily be done inline in the AWS SAM template. We add one policy to allow our shutdown function to stop our database instance and one policy to allow our startup function to start our database instance.

When we package and deploy, we can test our shutdown function again and we see that this time it does work.

Schedule shut down / start up

Our goal was to save some money, so we now need to make sure the database goes down when it won’t be used. We will shut it down between 7 PM and 7 AM on weekdays and completely in the weekend.

We do this by adding scheduled CloudWatch events as triggers for our functions. Don’t forget the cron timings need to be specified in UTC.

When we package and deploy, we see the correct CloudWatch events are added to our Lambda functions.

Monitoring

We can easily monitor the execution of our Lambda functions using CloudWatch, but we don’t want to go and check every day if the shutdown and/or startup was successful.

In fact, we only want to be bothered in case our functions fail. To do so, we will set up an AWS SNS topic that will act as a dead letter queue and link it to our Lambda functions.

In the AWS SAM template, we create the SNS topic with email subscription and reference it as the dead letter queue of our functions.

When we package and deploy again, we can see that the SNS topic was created.

We also receive an email prompting us to confirm our subscription to the newly created SNS topic. Once subscribed, we will receive an email in case one of our functions fails.

Clean up

A word of advice, always clean up any resources you aren’t using anymore. We are trying to save some money here, remember?

In order to delete the CloudFormation stack and all the resources it created, just run the command below.

Also don’t forget to delete the test database

and the S3 bucket with the deployment artifacts.

Final code

The final code of this project can be found on Github. You can use it directly by just modifying the database instance, ARN and shutdown / startup timings.
You can also use it as a basis to build similar functionality (e.g. automatic shutdown of EC2 instances) or to experiment a bit more with AWS SAM yourself.

In a next blog I’ll explain how to setup CI/CD for this project using AWS CodePipeline and other AWS Developer Tools.

Thanks for reading!

Maarten

HatchSoftware

Hatch Software’s company blog

Maarten Thoelen

Written by

Head of Technology @Hatch

HatchSoftware

Hatch Software’s company blog

More From Medium

More from HatchSoftware

More from HatchSoftware

Related reads

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade