Circle CI and DynamoDB

After searching around whilst building a Node.js API project, I realised there wasn’t too much documented on how to setup CircleCI with AWS DynamoDB for testing purposes within your build pipeline. I thought I’d do a quick post to summarise the process and hopefully make it clearer for others who are trying to achieve the same.

Getting Started

As I mentioned above, I was building a Node.js API, however this process should translate regardless of the language or tools you’re using. Before going any further, please make sure you’ve entered your AWS_ACCESS_KEY and AWS_SECRET_KEY into your CircleCI Project. This can be done from the setting page of your Project, under the Permissions heading.

Hit the AWS Permissions link and enter your variables

Now we’re going to need to edit our circleci config.yml file. The below is my finished version. I know it’s pretty long and messy and can definitely be refactored, but it does the job for now.

#YAML
version: 2
branches:
    only:
     - master # list of branches to build
jobs:
  build:
    docker:
      - image: circleci/node
working_directory: ~/repo
steps:
      - checkout
      - run:
          name: Install Java
          command: 'sudo apt-get update && sudo apt-get install default-jre default-jdk'
      - run:
          name: Install Python
          command: 'sudo apt-get update && sudo apt-get install -y python-dev'
      - run:
          name: Install Python
          command: 'sudo curl -O https://bootstrap.pypa.io/get-pip.py'
      - run:
          name: Install Python
          command: 'sudo python get-pip.py'
- run:
          name: Install AWS CLI
          command: 'sudo pip install awsebcli --upgrade'
      - run:
          name: Setup Container
          command: |
              curl -k -L -o dynamodb-local.tgz http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz
              tar -xzf dynamodb-local.tgz
              java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb
          background: true
      - run:
          name: Update yarn
          command: 'yarn global add npm@latest'
      - restore_cache:
          key: dependency-cache-{{ checksum "package.json" }}
      - run:
          name: Install Dependencies
          command: yarn install
      - save_cache:
          key: dependency-cache-{{ checksum "package.json" }}
          paths:
            - node_modules
      - run:
          name: Start Server
          command: 'yarn ci-start'
          background: true
      - run:
          name: Create Table
          command: 'yarn create-db'
      - run:
          name: Load Data
          command: 'yarn load-data'
      - run:
          name: Run Tests
          command: 'yarn test'
      - run:
          name: Deploy to AWS Elastic Beanstalk
          command: 'eb init MyApp -r eu-west-2 -p arn:aws:elasticbeanstalk:eu-west-2::platform/Node.js running on
                    64bit Amazon Linux/4.4.3'
      - run:
          name: Deploy to AWS Elastic Beanstalk
          command: 'eb deploy your-env'

First up we install Java as DynamoDB requires Java to run. The next part is optional, but as my app deploys to Elastic Beanstalk, I download Python as AWS EB CLI requires Python to run. Lastly we install DynamoDB directly from Amazon, I’ve chosen eu-west-2, but choose whichever location is nearest to you. This downloads as a zip, so we unzip it and then run the .jar file. The important thing here is to note the use of the option background: true. This ensures it runs in the background and doesn’t stall your build from going onto the next stage. From here, you can launch your server as a background task, load your data in and run your tests.

Hope you this helped anyone having trouble incorporating DynamoDB into their build pipelines. If you’re stuck, or have any questions, please ask!

As always, thanks for reading, hit 👏 if you like what you read and be sure to follow to keep up to date with future posts.