How to use DynamoDB Document Client with AWS Lambda DynamoDB Streams

Your DynamoDB Streaming code running in Lambda can have the same niceties of JS objects provided by the DocumentClient

DynamoDB has a cumbersome ‘type system’

When using DynamoDB, the API can take a little getting used to. One of the more annoying parts of the API is how attribute data types are specified. Each Item Attribute is itself a map with only one key, where the key signifies the datatype, like the following:

This means you can’t just access an attribute by saying item.attribute, you must say item.attribute.TYPE. Well, what if you don’t know the type? You could make the argument ‘This attribute is always a string, so let’s just assume it’s a string’, but you never know when someone in the future may change it’s type, either purposefully or in error.

AWS SDKs to the rescue!

Fortunately, most (all?) of the AWS SDKs attempt to ease some of this burden. Statically-typed languages like Java have added a layer on top of the raw API to do sensible things using the type system. Javascript has the ‘DocumentClient’ which automatically unmarshalls the DynamoDB data model into a more native-feeling javascript object.

Compare the two gists below:

With the Document Client

The above is way easier to manage than the ‘raw API’ as you see below.

Without the DocumentClient

Now this is all well and good, there is never a time when you wouldn’t use the DocumentClient …unless you’re using DynamoDB Streams

The trouble with using DynamoDB Streams

DynamoDB Streams is a feature where you can stream changes off your DynamoDB table. This allows you to use the table itself as a source for events in an asynchronous manner, with other benefits that you get from having a partition-ordered stream of changes from your DynamoDB table.

The problem is, when you use AWS Lambda to poll your streams, you lose the benefits of the DocumentClient! You are no longer calling DynamoDB at all from your code. Your Lambda is invoked with the body from the stream. You never are ‘given the chance’ to insert the DocumentClient in the right place to do this JS object translation for you.

The good news is … we can tap into this translator directly!

I did some digging and learned how to pull the nice object translation layer out of the DocumentClient and invoke it directly. See the gist below:

Use the DynamoDB DocumentClient object translator with AWS Lambda Stream Events

That’s all there is to it! Your DynamoDB Streaming code running in Lambda now has the niceties of JS objects as provided by the DocumentClient.

Some notes:

• This is in no way using public APIs from the SDK. These could change or go away at anytime. I protect from this issue by always including my own AWS SDK in my build — so Lambda SDK upgrades won’t break me — and thus my local integration tests will catch a SDK break.
• You may be able to use a single translateOutput call using the getRecords shape directly. I didn’t try very hard to make this work.

That’s it for this post. I hope this helps you cut down on a lot of the boilerplate in your DynamoDB Streaming listeners.

Happy Coding!