David Zhang
Nov 2, 2018 · 4 min read
Lambda + Aurora = ❤️

[Updated Nov. 22, 2018] Aurora Serverless accessible via HTTP!! (beta)

This story is part of a series to get started with Relational Databases in Serverless:


When getting started with serverless and AWS Lambda, DynamoDB is often the default choice to store data. And there is a reason for that. You want your database to scale with your functions without having to think about it:

Amazon DynamoDB is a nonrelational database that delivers reliable performance at any scale. […] (https://aws.amazon.com/dynamodb/)

But DynamoDB is a nonrelational database. Based on your needs, it can’t just replace a good old fashion MySQL or PostgreSQL 🙄 (see SQL or NoSQL)…

Why Aurora?

The promise of Aurora Serverless 😌! Scaling up and down with your Lambda functions:

Aurora Serverless is an on-demand, auto-scaling configuration for Aurora (MySQL-compatible edition) where the database will automatically start-up, shut down, and scale up or down capacity based on your application’s needs. […]

With some new features in development 🙏 🙏 🙏

  • Accessible over HTTPS — Currently in beta !! 😍 🎊 🔥 (see a first look here). As Jeremy describes it well in his article, it’s a great step forward to be able to use Aurora with Lambda, but we are not there yet, so I wouldn’t recommend using the Data API yet
  • Authentication & authorization over tightly-scoped IAM roles rather than database roles

I can confirm, that both features are being actively developed and initial timeframe of their release has not been changed. They will be released before end of 2018. Right now there is no exact release date. (Mateusz from AWS Premium Support, Sep 12, 2018)

Limits of this setup…

  • Aurora lives in a VPC and for a Lambda function to access it, it needs to live in the same VPC. Though, Lambda + VPC = 💔 because of its killer latency (see Nathan’s post: “VPC increased cold start times by an average of 8.83s”…)
  • While we wait for Aurora Serverless to become more mature, we need to manage connections to our database. Hopefully some serverless’ 👼 such as Jeremy Daly are here for you (see his excellent post and serverless-mysql package)
  • Migrations… Managing migrations with stateless functions… I love serverless, but sometimes, you got to re-invent the wheel… We’ll see how to deal with them in next articles 🙃

Mos case study

Knowing all this, at mos.com:

  • our front-end facing API is querying Aurora only to asynchronously inserts data (via SQS). That way, our users won’t experience Lambda + VPC’s 9s cold starts
  • data from Aurora is only used by asynchronous background jobs, as latency is not a big issue for them

Let’s get started!

We are using Serverless Framework (mainly because they got the serverless.com domain… 🙄)

serverless.yml (see gist below)

Resources needed:

  • VPC (AWS::EC2::VPC) and its Subnets (AWS::EC2::Subnet) where Aurora and Lambda will live
  • Aurora: Subnet Group (AWS::RDS::DBSubnetGroup); Cluster (AWS::RDS::DBCluster) and its Cluster Parameter (AWS::RDS::DBClusterParameterGroup); Instance (AWS::RDS::DBInstance) and its Instance Parameter (AWS::RDS::DBParameterGroup)
  • SQS (AWS::SQS::Queue) to trigger asynchronously our Lambda function

handler.js (see gist below)

  • We are using serverless-mysql package to handle connections to the database. Do not forget to await mysql.end() after you finished your query to clean up the connections.

Let’s test!

  • Deploy everything serverless deploy
  • Create table via one of your trigger:
curl --header "Content-Type: application/json" \
--request POST \
--data '{"query":"CREATE TABLE test (createdAt DATETIME DEFAULT CURRENT_TIMESTAMP)"}' \
http://your.endpoint/query
ORsend-message
--queue-url https://queue.url
--message-body '{"query":"CREATE TABLE test (createdAt DATETIME DEFAULT CURRENT_TIMESTAMP)"}'
  • You should be all set 🎊 🎈 🎺 !

What’s next?

As I do not encourage you to create endpoints through which you can execute any query on the database, we’ll see in the next article how to securely connect to the Aurora database through an EC2 instance. That way you’ll be able to directly query your database from your local machine.

Checkout the other stories of the series:

mos-engineering

Mission driven engineers fixing student financial aid. https://mos.com — We are hiring 🙃

David Zhang

Written by

mos-engineering

Mission driven engineers fixing student financial aid. https://mos.com — We are hiring 🙃

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade