AWS DynamoDB DocumentClient & Node.js — Complete Table Methods

In this blog post, I’ll take you through how to perform variety of operations starting from simple queries ending with complex transactions using AWS DynamoDB DocumentClient and Node.js.

There are two basic ways to interact with DynamoDB tables from Node.js applications:

• Class AWS.DynamoDB from AWS SDK for JavaScript.
• AWS.DynamoDB.DocumentClient which simplifies working with DynamoDB items by abstracting away DynamoDB Types and converting responses to native JS

This discussion mostly focus on DocumentClient but some of the operations like creating tables must be run using classical DynamoDB service.

Setup

Node.js application to work with DynamoDB is first make sure that aws-sdk is installed by using this command npm i aws-sdk — save. Once installed then copy the below code.

const AWS = require("aws-sdk") // Or use `import` syntax for newer ES versions

const dynamoDB = new AWS.DynamoDB({
  region: "us-east-1", // If not set, will get from ~/.aws directory or environment variable
  // and rest of properties
})

// Or

const documentClient = new AWS.DynamoDB.DocumentClient({
  region: "us-east-1",
  // and rest of properties
})

Create table

Database Structures data in tables, so if you want to save data to DynamoDB, first you need to create a table. You can do that using AWS-SDK for Javascript, like this

const AWS = require("aws-sdk")
const dynamoDB = new AWS.DynamoDB({ region: "us-east-1" })

dynamoDB
  .createTable({
    AttributeDefinitions: [
      {
        AttributeName: "id",
        AttributeType: "S",
      },
    ],
    KeySchema: [
      {
        AttributeName: "id",
        KeyType: "HASH",
      },
    ],
    BillingMode: "PAY_PER_REQUEST",
    TableName: "Enter your table name",
  })
  .promise()
  .then(data => console.log("Success!", data))
  .catch(console.error)

After this call resolves, it does not necessarily mean that table status is ACTIVE and it's is ready for read and write operations. Before start manipulating items in it, we should check if it's in ACTIVE state first using describeTable function ran every 5 seconds until it is ready:

const backoffInterval = 5000 // 5 seconds

const waitForTable = TableName =>
  dynamoDB
    .describeTable({ TableName })
    .promise()
    .then(data => {
      if (data.Table.TableStatus !== "ACTIVE") {
        console.log(
          `Table status: ${data.Table.TableStatus}, retrying in ${backoffInterval}ms...`
        )
        return new Promise(resolve => {
          setTimeout(() => waitForTable().then(resolve), backoffInterval)
        })
      } else {
        return
      }
    })
    .catch(error => {
      console.warn(
        `Table not found! Error below. Retrying in ${backoffInterval} ms...`,
        error
      )

      return new Promise(resolve => {
        setTimeout(() => waitForTable().then(resolve), backoffInterval)
      })
    })

waitForTable("my-table").then(() => console.log(`my-table is ready!`))

Get All Items / Scan in DynamoDB

After our table is provisioned and it’s in ACTIVE state, first thing that we probably would like to do is get all items. If you want to narrow your search results, use FilterExpressions combined with ExpressionAttributeNames object like so:

const AWS = require("aws-sdk")
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" })

dynamoDB
  .scan({
    TableName: "my-table",
    FilterExpression:
      "attribute_not_exists(deletedAt) AND contains(firstName, :firstName)",
    ExpressionAttributeValues: {
      ":firstName": "John",
    },
  })
  .promise()
  .then(data => console.log(data.Items))
  .catch(console.error)

You will in fact return all the items in the table under one condition. you have less than 1MB of data inside it.

Get Item

If you know the exact Partition Key (and Sort Key if using composite key) of the item that you want to retrieve from the DynamoDB table, you can use get operation:

const AWS = require("aws-sdk")
AWS.config.update({ region: "us-east-1" })
const dynamoDB = new AWS.DynamoDB.DocumentClient()

dynamoDB
  .get({
    TableName: "my-table",
    Key: {
      id: "123", // id is the Partition Key, '123' is the value of it
    },
  })
  .promise()
  .then(data => console.log(data.Item))
  .catch(console.error)

Batch Get Item

The BatchGetItem operation returns the attributes of one or more items from one or more tables. You identify requested items by primary key.

A single operation can retrieve up to 16 MB of data, which can contain as many as 100 items. BatchGetItem returns a partial result if the response size limit is exceeded, the table's provisioned throughput is exceeded, or an internal processing failure occurs. If a partial result is returned, the operation returns a value for UnprocessedKeys. You can use this value to retry the operation starting with the next item to get.

const AWS = require("aws-sdk")
AWS.config.update({ region: "us-east-1" })
const dynamoDB = new AWS.DynamoDB.DocumentClient()

dynamoDB
  .batchGet({
    RequestItems: {
      "my-table": {
        Keys: [
          {
            id: "123",
          },
          {
            id: "124",
          },
        ],
      },
      "other-table": {
        Keys: [
          {
            id: "abc",
          },
          {
            id: "abd",
          },
        ],
      },
    },
  })
  .promise()
  .then(data => console.log(data.Responses))
  .catch(console.error)

Put Item

put operation creates a new item, or replaces an old item with a new item if it's using the same key(s)

const AWS = require("aws-sdk");
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" });

dynamoDB
  .put({
    Item: {
      id: "156788",
      name: "skoda",
      email: "skoda@do.io",
    },
    TableName: "my-table",
  })
  .promise()
  .then((data) => console.log(data.Attributes))
  .catch(console.error);

Batch Write / Put Item

The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can transmit up to 16MB of data over the network, consisting of up to 25 item put or delete operations. While individual items can be up to 400 KB once stored

BatchWriteItem cannot update items. To update items, use the UpdateItem action.

const AWS = require("aws-sdk")
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region: "ap-south-1" })

dynamoDB
  .batchWrite({
    RequestItems: {
      MyTable: [
        {
          DeleteRequest: {
            Key: { id: "13456" },
          },
        },
        {
          PutRequest: {
            Item: {
              id: "456",
              name: "skoda",
              email: "tdanesh@dynamodb.in",
              description: "Professional DynamoDB Client",
            },
          },
        },
      ],
    },
  })
  .promise()
  .then(data => console.log(data.Attributes))
  .catch(console.error)

Query for a Set of Items

You can use the query method to retrieve data from a table. You must specify a partition key value; the sort key is optional.It also allows to use multiple operators for SortKey such as begins_with or mathematical ones like >, =, >= and so on.

const AWS = require('aws-sdk');
const dynamoDB = new AWS.DynamoDB.DocumentClient({region:'us-east-1'});

dynamoDB
  .query({
    TableName: 'your-table name',
    KeyConditionExpression: 'id = :hashKey and date > :rangeKey'
    ExpressionAttributeValues: {
      ':hashKey': '12345',
      ':rangeKey': 2015010112
    }
  })
  .promise()
  .then(data => console.log(data.Items))
  .catch(console.error);

Simple Transaction

DynamoDB also support transactions — they allow to run multiple write operations atomically meaning that either all of operations are executed successfully or none of them. It is especially useful when dealing with applications where data integrity is essential, e.g. in e-commerce — adding an item to a cart and decrementing count of items still available to buy.

Such flow should:

• Should happen atomically — these two operations should be treated as one, we don’t want to have a single moment in time where there’s a discrepancy in items count
• Should succeed only if count of items available to buy is greater than zero

Described flow can be modelled in Javascript & DocumentClient like this:

const AWS = require("aws-sdk")
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region:'us-east-1' })

await dynamoDB.transactWrite({
  TransactItems: [
    {
      Put: { // Add item to cart
        Item: {
          id: '1',
          count: '1'
        }
        TableName: "CartTable",
      },
    },
    {
      Update: { 
// decrement count of items available to buy only if the count is greater than zero
        ConditionExpression: "#count > 0",
        ExpressionAttributeNames: { "#count": "count" },
        ExpressionAttributeValues: {
          ":value": 1,
        },
        Key: {
          id: '12345',
        }
        TableName: "ItemsTable",
        UpdateExpression: "SET #count = :count - :value",
      },
    },
  ],
}).promise();

Read Transaction

Transaction can be also used for reading data atomically. Like in batchGet, you can retrieve the data from multiple tables in a single call:

const AWS = require("aws-sdk")
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region: "us-east-1" })

await dynamoDB
  .transactGet({
    TransactItems: [
      {
        Get: {
          TableName: "TableOne",
          Key: {
            HashKey: "abcd",
          },
        },
      },
      {
        Get: {
          TableName: "TableTwo",
          Key: {
            HashKey: "1234",
          },
        },
      },
    ],
  })
  .promise()

Query with Sorting

You can use the sort-key and apply the ScanIndexForward parameter in a query to sort in either ascending or descending order. Here I limit items returned to 1.However, if you need to sort DynamoDB results on sort key descending or ascending, you can use following syntax:

const AWS = require('aws-sdk');
const dynamoDB = new AWS.DynamoDB.DocumentClient({ region: 'ap-south-1' });

dynamoDB
  .query({
    TableName: 'my-table',
    IndexName: 'Index', // Main one
    KeyConditionExpression: 'id = :hashKey and date > :rangeKey'
    ExpressionAttributeValues: {
      ':hashKey': '12345',
      ':rangeKey': 2015010112
    },
    ScanIndexForward: true // true or false to sort by "date" Sort/Range key ascending or descending
  })
  .promise()
  .then(data => console.log(data.Items))
  .catch(console.error);

Thank you for reading!