How Provisioning an AWS Aurora Serverless v2 Database Cluster Differs from v1

Over the past year, I’ve had a growing interest in the infrastructure side of things on AWS. Recently, I took up the responsibility of provisioning relational databases for our team’s new microservices that would potentially be shipped to production.

I chose Aurora Serverless as this would manage database auto-scaling to match our workload. I whipped up a Cloudformation template for deployment which I quickly realised would create Aurora Serverless v1, not v2.

After looking into it, I decided on the shiny new toy. Why? Aurora Serverless v2 was made available by AWS in April 2022 and AWS added support for more features, including support for global databases, IAM database authentication, backtracking, replicas, and more granular scaling, to name a few.

Aurora Serverless v2 provides the full breadth of Amazon Aurora capabilities, including Multi-AZ support, Global Database, RDS Proxy, and read replicas.
- Amazon Web Services; Apr 27, 2022

Given that I previously created Aurora Serverless v1 databases, it should be pretty straightforward to create v2, right? Not really. I’ll talk about a few gotchas. First, let us look at how I provisioned Aurora Serverless v1.

Aurora Serverless v1 Cloudformation Resource

Here’s a barebones resource configuration:

V1Cluster:
  Type: AWS::RDS::DBCluster
  Properties:
    DatabaseName: ${param:DatabaseName}
    DBClusterIdentifier: ${param:DBClusterIdentifier}
    DBSubnetGroupName: ${param:SubnetGroupName}
    Engine: aurora-postgresql
    EngineMode: serverless
    EngineVersion: 10.18
    MasterUsername: ${param:MasterUsername}
    MasterUserPassword: ${param:MasterUserPassword}
    Port: 5432
    VpcSecurityGroupIds: 
      - ${param:SecurityGroupId}
    ScalingConfiguration:
      AutoPause: ${param:AutoPause}
      MinCapacity: ${param:MinCapacity}
      MaxCapacity: ${param:MaxCapacity}

The above template creates an Aurora Serverless v1 cluster with a Postgresql engine. Pay attention to the resource type which is AWS::RDS::DBCluster, typically used to create any RDS database cluster through Cloudformation. Also, I’ve passed my configuration as parameters and it is up to the deployer to determine and define your preferred configuration to match your needs.

Now, let us take a look at provisioning version 2.

Aurora Serverless v2 Cloudformation Resource

Here’s a barebones resource configuration:

V2Cluster:
  Type: Custom::DBCluster
  Properties:
    ServiceToken: !GetAtt ClusterLambdaFunction.Arn
    Version: 1
    DatabaseName: ${param:DatabaseName}
    DBClusterIdentifier: ${param:DBClusterIdentifier}
    DBSubnetGroupName: ${param:SubnetGroupName}
    EnableIAMDatabaseAuthentication: true
    Engine: aurora-postgresql
    EngineMode: provisioned
    EngineVersion: 14.3
    MasterUsername: ${param:MasterUsername}
    MasterUserPassword: ${param:MasterPassword}
    Port: 5432
    VpcSecurityGroupIds: ${param:SecurityGroupId}
    ServerlessV2MinCapacity: ${param:MinCapacity}
    ServerlessV2MaxCapacity: ${param:MaxCapacity}

This template creates an Aurora Serverless v2 cluster with a Postgresql engine and I’ve enabled IAM authentication, which is one of the perks of using v2. This resource has been defined to use a custom resource type, denoted as Custom::DBCluster, hence I have added a service token and version number. I’ll explain the need for these as we go.

We’ve seen how both resources have been defined, so let us take a look at those gotchas, which will explain why the above configurations for v2 and v1 look different.

Cloudformation Limitations

Although AWS Cloudformation supports Aurora Serverless v1, it does not currently support the creation of v2. A cluster for Aurora Serverless v2 can be created using AWS CLI or AWS RDS API, but it seems like the Cloudformation team may not have gotten around to enabling this feature yet. I decided to use a lambda-backed custom resource paired with the API as a workaround for provisioning my v2 database cluster.

Update: Amazon Aurora Serverless v2 now supports AWS CloudFormation

- Amazon Web Services; Oct 5, 2022

In the above template, I defined the service token, a required field that is typically used by custom resources to reference the lambda handling its events. I also defined a version number, an optional field that helps trigger updates if changes have been made only to the lambda code. A tip I picked up from my previous encounters with Cloudformation custom resources.

Within my handler, I instantiated an RDS client to create my database cluster. Here’s what that looked like:

import {
  RDSClient,
  CreateDBClusterCommand,
  CreateDBClusterCommandInput,
  CreateDBClusterCommandOutput
} from '@aws-sdk/client-rds';

export const createCluster = async ({ config }: any) => {
  const {
    DatabaseName,
    DBClusterIdentifier,
    DBSubnetGroupName,
    EnableIAMDatabaseAuthentication,
    Engine,
    EngineMode,
    EngineVersion,
    MasterUsername,
    MasterUserPassword,
    Port,
    VpcSecurityGroupIds,
    ServerlessV2MinCapacity,
    ServerlessV2MaxCapacity
  } = config;
  const client = new RDSClient({ region: process.env.AWS_REGION });
  const input: CreateDBClusterCommandInput = {
    DatabaseName,
    DBClusterIdentifier,
    DBSubnetGroupName,
    EnableIAMDatabaseAuthentication,
    Engine,
    EngineMode,
    EngineVersion,
    MasterUsername,
    MasterUserPassword,
    Port,
    VpcSecurityGroupIds: [VpcSecurityGroupIds],
    ServerlessV2ScalingConfiguration: {
      MinCapacity: ServerlessV2MinCapacity,
      MaxCapacity: ServerlessV2MaxCapacity
    }
  };
  const command = new CreateDBClusterCommand(input);

  try {
    const data: CreateDBClusterCommandOutput = await client.send(command);
    return data.DBCluster;
  } catch (error) {
    console.trace('Error creating database:', error);
    throw error;
  }
};

Lambda-backed custom resources let you pass properties as part of the event that your lambda consumes. If you’re not familiar with custom resources here is a guide to how they work.

I created a Lambda handler and passed in the configuration as resource properties. While defining the configuration, I realised a few more differences.

Configuration Differences

Engine Mode

Aurora Serverless v1 is configured by provisioning a database cluster with a serverless engine mode and this cluster does not need a database instance to be defined. A v2 database, on the other hand, is actually a provisioned database cluster, i.e a cluster with a provisioned engine mode, to which serverless database instances are attached.

Below is a barebones configuration of one of the instances. The instance class has to be set to db.serverless to provision Aurora Serverless v2 instances. It inherits scaling configuration amongst others from its cluster.

V2Instance:
   Type: AWS::RDS::DBInstance
   DependsOn: V2Cluster
   Properties:
     DBInstanceClass: db.serverless
     DBClusterIdentifier: !GetAtt V2Cluster.DBClusterIdentifier
     DBInstanceIdentifier: ${param:DBInstanceIdentifier}
     Engine: aurora-postgresql

Engine versions

Aurora Serverless v1 and v2 are supported by specific versions of Postgresql and Mysql. These versions differ, with v2 having mostly later versions of the engines. In the example above, I have specified Postgresql 10.18 and 14.3 for v1 and v2 respectively. I refer to the AWS RDS documentation in situations like this to make sure I have the correct specs.

Scaling Configuration

For Aurora Serverless databases, scaling configurations need to be defined. Note that for serverless v2, the property name has serverless v2 attached as a prefix. So while configuration for v1 is defined withScalingConfiguration, that of v2 is ServerlessV2ScalingConfiguration

And that's it on the list of things that stood out to me. If you’re thinking of using Aurora Serverless anytime soon, those are a few things to look out for.