Testing DynamoDB offline with Serverless Framework

Michael Timbs
Mar 10 · 4 min read

At some point in your serverless journey you’ll no doubt utilise DynamoDB. You’ll also probably want to be able to write integration tests against it. In this post we’ll go through the basics of getting an offline environment set up for testing, in a future post i’ll walk you through the couple of lines of code that needs to change if you want to utilise DynamoBBStreams (seperate post for SEO and maximum help for anyone debugging the issue).


Configuring serverless-dynamodb-local

We’ll begin by building on top of the serverless template I wrote about previously which can be found on github here. Whether you are using that template is irrelevant but I’ll assume you have a serverless project up and running already.

First thing we will pull serverless-dynamodb-local. This plugin will spin up a local instance of DyamoDB for us. Note: You’ll need to make sure you have JRE installed on your machine.

npm i -D serverless-dynamodb-local

Next we’ll add it to the plugins list (make sure it is above serverless-offline plugin)

plugins:
- serverless-dynamodb-local
- serverless-offline

We’ll also configure it to run during a test stage (jest sets the stage to test while running), allow it to run in memory and run migrations as well.

custom:
dynamodb:
stages:
- test
start:
inMemory: true
migrate: true

Creating the DynamoDB Resource

Some people will create infrastructure in a dedicated project, personally I prefer to create resources in services that own them. Best practices in serverless are still evolving so I tend to stick with the tried strategy of a seperate database (DynamoDB table) per service. I also subscribe to Rick Houlihan’s insistence of no more than one database per service.

With that preamble we can now spin up a DynamoDB table in code. In `servereless.yml`we can define the DynamoDB resource which will then be spun up and managed via CloudFormation.

resources:
Resources:
DynamoDBTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: ${self:service}-${self:provider.stage}-event-store
BillingMode: PAY_PER_REQUEST
KeySchema:
- AttributeName: pk
KeyType: HASH
- AttributeName: sk
KeyType: RANGE
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
- AttributeName: sk
AttributeType: S

Configuring Jest

Ok we now have DynamoDB spun up as a resource for our project and have the ability to interact with it offline for testing. Now we just need to let Jest know how to work with it.

Firstly we need to install the following package

npm i -D @shelf/jest-dynamodb

In the base template I am building from I already have Jest installed as well as a jest config file setup.

We need to add the following line of code underneath the setupFiles key in our jest config file.

preset: '@shelf/jest-dynamodb',

Now we need to create a jest-dynamodb-config file. There are many ways to do this but my preferred way is to read straight from our serverless.yml and create the table fro the resources we created there.

The following code will grab all of the resources defined in your serverless template, filter for DynamoDB tables and then grab all the properties on it and create it and make it accessible locally on port 8000;

jest-dynamodb-config.js

Write code that is easy to test

Having Dynamo offline configured isn’t going to do much good if we can’t easily adapt our code to use it if in an offline environment.

My preferred method of doing this is to create a base DynamoDBClient that I import from in all my modules. Then I can do the logic for instantiating online/offline clients in one place instead of having to do it all through my code.

If you are using the @dazn/lambda-powertools-dynamodb-client that I recommended in my previous article, this base module would look like this

DynamoDB base client with dazn powertools

If you are not using that library you would do something like this

DynamoDB base client with standard DocumentClient

Define a custom table name in Jest Environment

You can define a set of actions to perform before your jest test runs. I use this setup to set/overwrite environment variables. This is my base template

Overwriting the environment variables in tests

All we need to do is add an environment variable for our DynamoDB table name (if required). Right at the start I named our table

TableName: ${self:service}-${self:provider.stage}-event-store

Unfortunately the stage defaults to local in my default template. So for this specific naming convention I would want to set something like

DYNAMODB_TABLE: 'typescript-serverless-local-event-store'

Wrapping Up

That’s all there is to it. You should be able to start writing integration tests against the database. As I mentioned earlier in the article you will need to tweak this config just ever so slightly to deal with DynamoDB streams if you want to use them. I’ll write about that tweak very soon and update this section with a link when I have.

Michael Timbs

Written by

Full-stack polyglot Software Developer at fleet.space, Pretend Data Scientist.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade