How I Built a Serverless BigCommerce App on AWS

Patrick Puente
BigCommerce Developer Blog
38 min readSep 24, 2019

Introduction

Serverless applications are cool. A serverless application can scale out of the box. It can be cost effective since you only pay for what you use. And, you sound really smart when you throw around cool words like “serverless”. At least, I like to think I do.

Today AWS, Google Cloud, and Microsoft Azure compete in the serverless computing space. If you’re building a serverless application, choose the one that’s right for you. For me, I’ve been learning about AWS, so I chose AWS. If I had been learning about any other serverless compute providers, I would probably have used them.

Why to do the thing

I am lazy. That makes the idea of letting someone else manage provisioning, replica sets, and hosting pretty appealing. And I’m cheap, which means I am super interested in the AWS Free Tier offering. Now, while my first project was developed and tested within the bounds of the Free Tier, it was certainly possible to incur costs with the AWS resources I created. In fact, I came dangerously close to spending $0.01 when, by redeploying my cloud resources probably more often than I needed, I went over my 2,000 free put requests to S3.

Edit: After a second month of working with a few serverless BigCommerce apps, I did end up incurring less than $1 of charges from Secrets Manager when my 30 day trial expired. The cost for storing and accessing BigCommerce Client Secrets in Secrets Manager for a production-scale app can be relatively predictable. Before taking any app to production, it’s a good idea to get familiar with the AWS pricing models like Secrets Manager pricing.

What does the thing do?

Screenshot from the example app
Screenshot of the thing in the BigCommerce control panel

The thing generates a BigCommerce API token on behalf of a store owner, stores it, and uses it to retrieve information about the store from the BigCommerce API. It also provides a UI in the store’s admin panel where a user can use it. But, most importantly, it may give you a place to start. The example app in this article was the result of the lessons learned while building a little utility to make my life a little easier.

I had been testing BigCommerce webhooks quite a bit. Typically, I would use Postman to send the requests to the BigCommerce API, and ngrok to receive the webhook events at my local server. But, like I said, I’m cheap. So, using the free version of ngrok, my webhook destination URLs were never permanent. I had to set up ngrok and a new webhook each day, and tear them down at the end of the day. That was getting tiresome.

So, why not build a Webhooks Manager utility that would allow me to set a webhook from a UI in my BigCommerce control panel, and retain the same destination URLs? Inspired by a previous BigCommerce project, I used Requestbin to maintain my webhook destination URLs and event storage. You can find the end result here.

What is the thing?

The rest of this article will be my attempt to explain how I got from my silly idea for a utility to make my job easier to a functional serverless BigCommerce app. And, hopefully, it will save you some time. To skip straight to the end and start using the thing…

…here’s a link to the example app.

Serverless applications are just that, applications that don’t need servers. Okay, fine, obviously they need servers. But, the servers are abstracted so far away from the cloud resources that make up the application that you don’t have to think about them. Imagine spending your time building a cool app without worrying about provisioning VMs, configuring load balancers, or managing failovers. Yes, there are stoic ops teams around the world doing exactly that kind of work in the shadows every day to make our beloved internet a reality, and they are all heroes. But I am no hero. I just wanted to build a cool app.

In AWS, this app uses a few types of serverless resources. They are Lambda functions for serverless compute, DynamoDB for a NoSQL database, Cognito for customer information management and authentication, API Gateway to authenticate API requests against Cognito and route them to the appropriate resources, and S3 for hosting. Each of these services will be explored later in this article.

Prior knowledge that was helpful

AWS

Having done some online AWS training, I was familiar enough with the names of services to tell you in general terms what they do. But, this was my first attempt at deploying an application on AWS. So, a little bit of prior knowledge of AWS is helpful, but you don’t need to be a guru. To get started, there are a couple of important concepts to understand.

  • IAM
    AWS Identity and Access Management (IAM) allows you to securely grant and deny access to resources in AWS. AWS Amplify will use IAM roles to grant access to resources in your project. For example, if a Lambda function needs read access to a table in DynamoDB, the Lambda can be granted an IAM role with read permissions for the table.
  • CloudFormation
    AWS CloudFormation allows you to create templates AWS can use to programmatically deploy your resources. It, and similar services, are often described as “infrastructure as code”.

Throughout this project, you will manually modify a few CloudFormation templates to do things like configure environment variables and grant access permissions to other resources in the project. AWS Amplify CLI does a great job of automating access permissions for resources it provisions, but Amplify CLI does not support all of the AWS services this project uses. So, for those services like Secrets Manager that Amplify doesn’t support, you will need to manually edit the CloudFormation templates the CLI creates. There are more details on that process later in this article.

NodeJS

AWS Amplify JS, the Amplify CLI, the frontend framework and the Lambda functions that provide the compute resources for the app all use the NodeJS runtime environment. A working knowledge of NodeJS and npm is a must.

A Frontend Framework

Amplify JS has built-in support for a few frontend frameworks including React, React Native, Angular, and Vue. The example app is built with React.

It may have been my preference, or maybe just plain ignorance, but I didn’t find the prebuilt components in aws-amplify-react to lend themselves well to the BigCommerce app installation and authentication flows. Instead, the app uses aws-amplify to create a Cognito client that manages authentication.

Express.js

Yes, yes, I said this would be serverless. So why does knowing Express help? Because AWS uses a version of Express called aws-serverless-express in some Lambda functions. For this project, the app’s API and Uninstall Callback URL both use aws-serverless-express.

A NoSQL Database

Before this project, I had never used DynamoDB. But, I did have some minimal experience with MongoDB, and that experience constructing MongoDB queries helped when it came to constructing DynamoDB queries. Experience with some type of a NoSQL database can be helpful, but is not entirely necessary.

REST APIs

The BigCommerce APIs this app consumes are REST APIs. Knowledge of the BigCommerce APIs, or at least where to find the documentation for the endpoints you want to consume, is essential.

The application provides its own REST API to access the backend Lambda resources needed to interact with BigCommerce. Some familiarity with AWS API Gateway may be helpful, but was not necessary.

BigCommerce APIs and App Installation Flows

The point of the app is to interact with BigCommerce via APIs, so knowing what those APIs are, how to authenticate against them, and how to obtain the necessary authentication tokens is paramount. If none of this sounds familiar, stop here and check out the BigCommerce API Quick Start. Once you’re comfortable running requests against the BigCommerce API from a REST client or the developer portal, come back and build a cool serverless app.

  • Installation Flow
    The installation flow will be covered in detail later. If you’re new to this, check out the App Installation and Update Sequence in the Building an App guide. For now, it’s important to understand that only the user in the BigCommerce store with store owner permissions is authorized to trigger the installation flow.
  • Authentication Flow
    The authentication flow will be covered in detail later. If you’re new to this, check out the Load, Uninstall, and User Removal Requests and the Processing Signed Payload sections in the Building an App guide.

It’s worth noting that the BigCommerce installation and authentication flows informed the account structure used by the app. While there may be many user accounts in a BigCommerce store, individual users do not authenticate with the app. Instead, BigCommerce uses an OAuth flow to authorize the app’s access to BigCommerce, and a secret to sign tokens that grant authorized users access to the app.

BigCommerce apps can optionally enable multi-user support. If an app does not enable multi-user support, then only the BigCommerce user with store owner permissions is able to trigger the Load Callback Request, so only one user at a time will ever access the app for that store. I say “at a time” because the role of store owner can be reassigned, which I’ve interpreted to mean that the store is the app user while the store owner is an attribute of the user.

If an app does enable multi-user support, then any user in that store may be granted permissions to use the app. There is no additional signup process for the app to manage when a new user opens the app for the first time. Instead, the app simply receives a request at the Load Callback URL with a new user’s information. So, upon validating the request’s signature, the new user should have access to the app.

So, I made the decision to create a user in the Cognito user pool for each store, rather than each BigCommerce user account that may access the app. The app does store the BigCommerce store owner’s username in DynamoDB, and it doesn’t store information for any other users that may load the app if multi-user support is enabled. It also doesn’t update the store owner information in DynamoDB if a Load request identifies a new store owner. A production-grade application should consider these use cases, and account for new users and new store owners in the context of a single BigCommerce store.

Personal Note

The primary objective of this project was learning. I am working to understand AWS services and best practices, and a project like this was a great way to get started. But, of course, mistakes were made. I will attempt to point out the limitations of the application wherever possible. For instance, some of the steps that require accessing the AWS console could be automated with the use of the AWS CLI/SDK. Further, there is no testing suite, or CI pipeline. In short, there is plenty of room for improvement. Consider this your disclaimer: this article is an AWS noob’s journey to building a serverless app.

Start Building the Thing

Local Environment

Install and Configure Amplify CLI

If you don’t already have an AWS account, create one now. You will need to set up a payment method. While I strove to keep the resources this app requires within the Free tier thresholds, it is certainly possible to incur charges by using the AWS resources described in this guide. It’s a good idea to set up a CloudWatch alarm to notify you when your estimated AWS charges exceed a certain threshold. I’m cheap, so I set mine to $5.

Before going further, this is a friendly reminder to always follow AWS IAM best practices like operating out of an IAM user instead of your root account.

Follow the Get Started instructions to install and configure Amplify CLI:

$ npm install -g @aws-amplify/cli
$ amplify configure

React Client

Create React App
The frontend is bootstrapped with create-react-app. From the directory where you want to create your project folder, run:

$ npx create-react-app my-app
$ cd my-app

Feel free to name your project folder something other than my-app. For more information see the Create React App documentation.

Initialize Amplify
Run amplify init. It’s a deceptively simple command. To quote the AWS documentation:

The init process
$ amplify init

The init command must be executed at the root directory of a project to initialize the project for the Amplify CLI to work with. The init command goes through the following steps:

- Analyzes the project and confirms the frontend settings
- Carries out the initialization logic of the selected frontend
- If there are multiple provider plugins, prompts to select the plugins that will provide accesses to cloud resources
- Carries out, in sequence, the initialization logic of the selected plugin(s)
- Insert amplify folder structure into the project’s root directory, with the initial project configuration
- Generate the project metadata files, with the outputs of the above-selected plugin(s)

Source: https://aws-amplify.github.io/docs/cli-toolchain/quickstart

In your project folder, run:

$ amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project my-app
? Enter a name for the environment dev
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path: src
? Distribution Directory Path: build
? Build Command: npm run-script build
? Start Command: npm run-script start
Using default provider awscloudformation
For more information on AWS Profiles, see:
https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
? Do you want to use an AWS profile? Yes
? Please choose the profile you want to use default
⠋ Initializing project in the cloud…

AWS Amplify

The React app will access the resources created by Amplify CLI with aws-amplify.

$ npm install - save aws-amplify

Import aws-amplify into src/App.js:

In App.js, configure Amplify with the src/aws-exports.js file that Amplify CLI generated:

React Router DOM

The routing in the app is handled by React Router DOM. You may use any routing library you prefer, but that is outside the scope of this article.

$ npm install --save react-router-dom

Note: If you encounter an EACCES or similar error when trying to rename or delete a module in ./node_modules, try deleting the ./node_modules folder, then running npm install.

BigCommerce App Registration

This example will walk you through registering a personal app. To register a personal BigCommerce app, you will need:

  1. Navigate to https://devtools.bigcommerce.com and log into your BigCommerce account with store owner permissions
  2. Click the Create an App button
  3. Enter your App Name, then click Create
  4. Navigate to the Technical tab then enter the Auth Callback URL and Load Callback URL
  5. Select the OAuth scopes your app will require (learn more about BigCommerce OAuth scopes). This example assumes you select at least the read-only scope for Information & Settings
  6. Save the app
  7. Click the View Client ID link to view your Client ID and Client Secret. Take note of those values for use in Lambda environment variables and storage in Secrets Manager respectively.

For more information on creating BigCommerce apps, check out the BigCommerce documentation.

Secrets Manager

Verifying the authenticity of the Auth and Load callback requests requires a BigCommerce Client Secret generated by BC, and stored by the application. The client secret could be stored as an environment variable for the Lambda functions that access it, but AWS Secrets Manager provides an additional layer of security by preventing exposure of the secret in repositories, local environments, or the Lambda console.

Amplify CLI doesn’t provide tools to create or manage secrets with Secrets Manager. So instead, you can store it via the Secrets Manager console or AWS CLI and access it from Lambda functions with appropriate IAM policies. There will be an example of how to modify the Lambda function’s CloudFormation template to apply the appropriate IAM policies under PreSignup Cognito Trigger below.

Store your BigCommerce client secret in AWS Secrets Manager

  1. Log in to AWS console
  2. Click Services, and locate Secrets Manager
  3. Click the Store a new secret button
  4. Select Other type of secrets (e.g. API key)
  5. Enter a Secret key/value pair where the key is an arbitrary key name like client_secret, and the value is the BigCommerce Client Secret.
    You can retrieve your Client Secret from https://devtools.bigcommerce.com
  6. Take note of your key to use as an environment variable in CloudFormation templates later
  7. Choose an encryption key, or use the default key
  8. Click Next
  9. Enter a Secret name (example: dev/bc-secret), and optionally provide a description and/or tags
  10. Take note of the secret name to use later as an environment variable in CloudFormation templates
  11. Click Next
  12. Select Disable automatic rotation
  13. Click Next
  14. Review the configuration, then click Store
  15. In the Secrets Manager console, click the name of your secret to locate the Secret ARN
  16. Take note of the Secret ARN to use later an environment variable in CloudFormation templates.
    example format:
    arn:aws:secretsmanager:{region}:{account}:secret:{secret-name}

For more information, see Secrets Manager documentation.

Auth Callback

When a BigCommerce user with store owner permissions clicks the Install button for your application, BigCommerce will send a GET request to your app’s Auth Callback URL. The request includes code, scope, and context parameters, which the app will exchange for the permanent BigCommerce API token for that store. For more details on the code exchange, see the BigCommerce documentation.

The example app doesn’t support the Update flow. The Update flow is essentially the same as the Auth flow, but for currently authenticated apps that need updated scopes. If the app receives an Auth request for a store that has already installed the app, Cognito will return a “user already exists” error. If an app ever needs to adjust its the scopes it accesses in production, it should be able to handle Auth requests for users that already exist in Cognito.

In this example, the Auth Callback URL is a route in the React client. The React client will initiate the signup flow in a Cognito user pool. Part of that signup flow will require a Lambda function to handle the Cognito PreSignup Trigger. The PreSignup Lambda is responsible for:

  • Exchanging the temporary code from the GET request for a permanent API token
  • Storing the permanent API token
  • Accepting or deny the Cognito signup request
Auth callback sequence diagram
Auth callback sequence

As you use Amplify CLI to build out your backend resources that complete the signup flow, it may help to create them in the reverse order in which they are invoked. That way, when you finish creating one resource, you can grant access permissions to the next resource you create. If you need to go back and modify permissions later, you can use the amplify <resource> update command to modify permissions for a resource you previously created with amplify <resource> add.

retrieveSecret Lambda Function

Several Lambda resources will need access to the BigCommerce Client Secret stored in Secrets Manager, including the PreSignup Lambda function Cognito will use in the auth flow. Unlike other resources like DynamoDB, Amplify CLI does not provide a handy tool for managing IAM permissions to access Secrets Manager. Instead, you must edit the PolicyDocument in the CloudFormation template, which will require information like the Secret’s ARN, and the Secret’s name. Instead of manually editing JSON for every Lambda that needs access to the Secret, it makes sense to use the CLI to create a single Lambda function that is responsible for accessing the Secret.

Create the retrieveSecret Lambda function with Amplify CLI:

$ amplify function add
Using service: Lambda, provided by: awscloudformation
? Provide a friendly name for your resource to be used as a label for this category in the project: retrieveSecret
? Provide the AWS Lambda function name: retrieveSecret
? Choose the function template that you want to use: Hello world function
? Do you want to access other resources created in this project from your Lambda function? No
? Do you want to edit the local lambda function now? No
Successfully added resource retrieveSecret locally.

Modify the CloudFormation Template
To grant the Lambda access to Secrets Manager, you must edit the PolicyDocument in the CloudFormation template, which will require information like the Secret’s ARN, and the Secret’s name. Additionally, the Secrets Manager client in the Lambda will need to know the key you set when you created the Secret’s key value pair. You can add environment variables for the key, and Secret name to the CloudFormation template. For portability, you can add the environment variables to a parameters.json file referenced by the CloudFormation template.

1. parameters.json
Create a file at amplify/backend/function/retrieveSecret/parameters.json with the following content:

Replace the values:

  • SECRETNAME: Your Secret’s name in Secrets Manager (example: dev/bc-secret)
  • SECRETKEY: The key name from the key value pair in Secrets Manager (example: client_secret)
  • SECRETARN: The Amazon Resource Name of your Secret in Secrets Manager (example format: arn:aws:secretsmanager:{region}:{account}:secret:{secret-name})

2. CloudFormation Template Parameters
In the amplify/backend/function/retrieveSecret/retrieveSecret-cloudformation-template.json CloudFormation template, add references to the Parameters object for the values in parameters.json:

3. PolicyDocument
In amplify/backend/function/retrieveSecret/retrieveSecret-cloudformation-template.json add permissible actions for Secrets Manager by adding an object to Resources.LambdaFunction.lambdaexecutionpolicy.Properties.PolicyDocument.Statement

4. Environment Variables
To make the values in parameters.json available as environment variables in the Lambda, add the following objects to Resources.LambdaFunction.Environment.Variables in amplify/backend/function/retrieveSecret/retrieveSecret-cloudformation-template.json

5. Write the Lambda Function
The function uses the SecretsManager class from the AWS SDK for JavaScript to return the secret.

check out the retrieveSecret Lambda Function code here

DynamoDB

DynamoDB serves as the storage for store information. The app will exchange the temporary code from the Auth Callback request for permanent API tokens, then store those tokens in DynamoDB. But first, you need to create the table in DynamoDB. For this example, I called it DynamoStores. It will have the following columns:

Hash

  • primary key
  • the BigCommerce store hash that uniquely identifies this store
  • retrieved from the context parameter in the Auth request
  • also used as Username in Cognito

Token

  • BigCommerce API token

Created

  • date created as a Unix epoch time

Modified

  • date modified as a Unix epoch time
$ amplify storage add
? Please select from one of the below mentioned services NoSQL Database
Welcome to the NoSQL DynamoDB database wizard
This wizard asks you a series of questions to help determine how to set up your NoSQL database table.
? Please provide a friendly name for your resource that will be used to label this category in the project: DynamoStores
? Please provide table name: DynamoStores
You can now add columns to the table.? What would you like to name this column: Hash
? Please choose the data type: string
? Would you like to add another column? Yes
? What would you like to name this column: Token
? Please choose the data type: string
? Would you like to add another column? Yes
? What would you like to name this column: Scope
? Please choose the data type: string
? Would you like to add another column? Yes
? What would you like to name this column: Created
? Please choose the data type: number
? Would you like to add another column? Yes
? What would you like to name this column: Modified
? Please choose the data type: number
? Would you like to add another column? No
Before you create the database, you must specify how items in your table are uniquely organized. You do this by specifying a primary key. The primary key uniquely identifies each item in the table so that no two items can have the same key. This can be an individual column, or a combination that includes a primary key and a sort key.To learn more about primary keys, see:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey
? Please choose partition key for the table: Hash
? Do you want to add a sort key to your table? No
You can optionally add global secondary indexes for this table. These are useful when you run queries defined in a different column than the primary key.
To learn more about indexes, see:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.SecondaryIndexes
? Do you want to add global secondary indexes to your table? No
? Do you want to add a Lambda Trigger for your Table? No
Successfully added resource DynamoStores locally

Cognito

Cognito fills both Customer Information Management (CIM) and Authentication & Authorization (A&A) roles. Because the app uses BigCommerce secrets and tokens rather than user-supplied usernames and passwords for authorization, custom Lambda triggers will need to be configured for the signup and auth flows. You will use Amplify CLI to provision the Cognito user pool, and to create a scaffolding for the following Lambda functions:

  • PreSignup
  • Create Auth Challenge
  • Define Auth Challenge
  • Verify Auth Challenge Response

The CLI will prompt you to enter an answer for the custom auth challenge. Enter an arbitrary non-secret string. The custom auth flows will not use this answer, and it will be visible in repos in amplify/team-provider-info.json unless you add that file to .gitignore.

$ amplify auth add
Using service: Cognito, provided by: awscloudformation

The current configured provider is Amazon Cognito.

Do you want to use the default authentication and security configuration? Manual configuration
Select the authentication/authorization services that you want to use: User Sign-Up, Sign-In, connected with AWS IAM controls (Enables per-user Storage features for images or other content, Analytics, and more)
Please provide a friendly name for your resource that will be used to label this category in the project: CognitoStores
Please enter a name for your identity pool. StoresIdentities
Allow unauthenticated logins? (Provides scoped down permissions that you can control via AWS IAM) No
Do you want to enable 3rd party authentication providers in your identity pool? No
Please provide a name for your user pool: StoresUsers
Warning: you will not be able to edit these selections.
How do you want users to be able to sign in? Username
Multifactor authentication (MFA) user login options: OFF
Email based user registration/forgot password: Disabled (Uses SMS/TOTP as an alternative)
Please specify an SMS verification message: Your verification code is {####}
Do you want to override the default password policy for this User Pool? No
Warning: you will not be able to edit these selections.
What attributes are required for signing up?
Specify the app's refresh token expiration period (in days): 1
Do you want to specify the user attributes this app can read and write? No
Do you want to enable any of the following capabilities? Custom Auth Challenge Flow (basic scaffolding - not for production)
Do you want to use an OAuth flow? No
? Do you want to configure Lambda Triggers for Cognito? Yes
? Which triggers do you want to enable for Cognito Create Auth Challenge, Define Auth Challenge, Pre Sign-up, Verify Auth Challenge Response
? What functionality do you want to use for Create Auth Challenge (Press <space> to select, <a> to toggle all, <i> to invert selection)Custom Auth Challenge Scaffolding (Creation)
? What functionality do you want to use for Define Auth Challenge (Press <space> to select, <a> to toggle all, <i> to invert selection)Custom Auth Challenge Scaffolding (Definition)
? What functionality do you want to use for Pre Sign-up Create your own module
? What functionality do you want to use for Verify Auth Challenge Response (Press <space> to select, <a> to toggle all, <i> to invert selection)Custom Auth Challenge Scaffolding (Verification)
? Enter the answer to your custom auth challenge This answer is arbitrary and will not be used by the app
Succesfully added the Lambda function locally
? Do you want to edit your boilerplate-create-challenge function now? No
Succesfully added the Lambda function locally
? Do you want to edit your boilerplate-define-challenge function now? No
Succesfully added the Lambda function locally
? Do you want to edit your custom function now? No
Succesfully added the Lambda function locally
? Do you want to edit your boilerplate-verify function now? No
Successfully added resource CognitoStores locally

PreSignup Lambda Function

After the React Client invokes Cognito to add the app to a new store, Cognito triggers the PreSignup Lambda to accept or deny the request. The Lambda is responsible for exchanging the temporary code supplied in the Auth request for a permanent API token, storing the token and store information in DynamoDB, and confirming or rejecting the signup request. In a larger-scale application, consider creating a separate Lambda that is solely responsible for DynamoDB operations to maintain separation of concerns and allow future resources to have a standard way of accessing the database.

Access to DynamoDB
The Amplify CLI created the local resources for the PreSignup Lambda when you added Cognito, but it did not grant the Lambda permissions to store data in DynamoDB. To add those permissions, use the CLI to edit the function:

$ amplify function update
Using service: Lambda, provided by: awscloudformation
? Please select the Lambda Function you would want to update CognitoStoresPreSignup
? Do you want to update permissions granted to this Lambda function to perform on other resources in your project? Yes
? Select the category storage, function
Storage category has a resource called DynamoStores
? Select the operations you want to permit for DynamoStores create, update
? Function has 2 resources in this project. Select the one you would like your Lambda to access retrieveSecret
? Select the operations you want to permit for retrieveSecret read
You can access the following resource attributes as environment variables from your Lambda function
var environment = process.env.ENV
var region = process.env.REGION
var storageDynamoStoresName = process.env.STORAGE_DYNAMOSTORES_NAME
var storageDynamoStoresArn = process.env.STORAGE_DYNAMOSTORES_ARN
var functionRetrieveSecretName = process.env.FUNCTION_RETRIEVESECRET_NAME
? Do you want to edit the local lambda function now? No
Successfully updated resource

Take note of the environment variables the CLI made available for the Lambda function. You will supply them to a DynamoDB Document Client and the Lambda client.

Find amplify/backend/function/CognitoStoresPreSignup in your project folder. You will edit the code in CognitoStoresPreSignup-cloudformation-template.json to add environment variables for the BigCommerce Client ID and Auth Callback URL, and you will edit code in the src folder to implement the signup flow.

Modify the CloudFormation Template

1. parameters.json

Replace the content of amplify/backend/function/CognitoStoresPreSignup/parameters.json with the following:

Replace the values:

Note: for better or worse, the custom.js file that Amplify CLI provides will not be used in this example. Instead, all code will be added to the index.js file. So, this example does not include the custom module in parameters.json.

2. CloudFormation Template Parameters

In the amplify/backend/function/CognitoStoresPreSignup/CognitoStoresPreSignup-cloudformation-template.json CloudFormation template, add keys to the Parameters object for the values in parameters.json:

3. Environment Variables

The Lambda will need access to additional environment variables for the POST request to https://login.bigcommerce.com/oauth2/token. You can add environment variables through the Lambda console after it is created, or add them to the CloudFormation template now.

To add them to the CloudFormation template now, add keys to Resources.LambdaFunction.Environment.Variables in amplify/backend/function/CognitoStoresPreSignup/CognitoStoresPreSignup-cloudformation-template.json:

4. Write the Lambda Function
It’s important to note that Cognito will retry the PreSignup event if the Lambda does not respond within 5 seconds. To prevent multiple attempts to exchange the same temporary code for a BigCommerce API token, the PreSignup Lambda will return a promise, which will resolve or reject depending on the outcome of the asynchronous network requests and DB operation.

A note about security
DynamoDB automatically encrypts all data at rest. But, without additional encryption in-transit, the tokens are at risk for exposure in the DynamoDB console. To mitigate the risk, this app base64 encodes API tokens before storage. Depending on your security requirements, you may consider using DynamoDB Encryption Client to encrypt the tokens client-side (in the Lambda) before storage.

A personal note
The reason I didn’t implement encryption with DynamoDB Encryption Client is that I don’t know Python or Java well enough to use the SDKs. AWS does not yet provide a JS SDK for DynamoDB Encryption Client. I suppose it’s about time to pick up some Python.

check out the PreSignup Lambda Function code here

React Client: Add Auth Callback Route

The Auth Callback route will receive a GET request from the user’s browser. The request includes a code parameter that the app will exchange with BigCommerce for a permanent API token.

Before submitting the code to the app backend or adding the user to Cognito, the Auth Callback route will prompt the user to accept the terms and conditions of the app. This route could also render a form that collects additional information about the user to store at attributes in Cognito.

Enable HTTPS
BigCommerce requires all app Auth and Load Callback URLs to use HTTPS. To enable HTTPS in the React client, add a .env file to the root directory of your project with the following:

BigDesign
BigCommerce provides a library of React components that lets developers build stylish apps with a native BigCommerce feel at their core. The example uses components from BigDesign, so the resulting app looks and feels like an extension of BigCommerce, rather than an unrelated add-on. You may use any component library you choose, but that is outside the scope of this article.

Run the following command from the project directory to install BigDesign and its dependencies:

$ npm install --save @bigcommerce/big-design @bigcommerce/big-design-icons styled-components

Route
The example app uses react-router-dom to create a route to /oauth in src/App.js. You may use any router you choose, as long as the resulting route matches the Auth Callback URL you supplied to BigCommerce.

…check out the routing in App.js here.

Install Component
The Install component is rendered by the route in App.js. It is responsible for:

  • collecting additional information, like an agreement for TOS or a privacy policy
  • invoking the signup flow in Cognito when the user agrees to sign up
  • displaying the result of the signup attempt
Screenshot of the install component
Screenshot of the Install component

When the user clicks the Sign Up button, the component uses the signUp() method of the Auth class from aws-amplify to initiate the signup flow in Cognito.

A note about security
To ensure the React client is only able to authenticate with the custom auth flow, be sure to enable Only allow Custom Authentication (CUSTOM_AUTH_FLOW_ONLY) for the app in the App Clients section of the Cognito User Pool console. See the Security Considerations section of the AWS blog post Customizing Amazon Cognito User Pool Authentication Flows.

Nevertheless, the signUp() method requires both a username and password. To ensure the password is not something easily guessable, the React client randomly generates a password between 16 and 256 characters including both alphanumeric and special characters.

check out the Install Component code here

Testing App Installation

Deploy the backend
Now is a good time to start deploying the backend to the cloud and testing. To deploy the resources you’ve added locally so far, run the following from the root directory of the project:

$ amplify serve

When you are prompted to create the resources in the cloud, answer Y. The amplify serve command compares your local backend to the backend in the cloud. If it detects any changes, it prompts to create, update, or delete the affected resources in the cloud. After deployment finishes, the React client’s local development server starts at which point the frontend is accessible on localhost.

You will receive a security warning in your browser because the app is served over HTTPS without an appropriate SSL certificate. The warning will be resolved when the React client is deployed to AWS hosting behind CloudFront with HTTPS. For now, follow the prompts to dismiss the warning and display the page.

Test Installation Flow

  1. Log in to your BigCommerce store. If you don’t have one yet, sign up for a trial. Make sure to use the same email address that you used to register the app at https://devtools.bigcommerce.com/.
  2. Navigate to Apps > My Apps > My Draft Apps
  3. Click your app’s name
  4. Click the Install button, and confirm installation

At this point, BigCommerce creates an iframe in the control panel that sends the Auth request to the Auth Callback URL. The Install component at that route should present the signup form. When the user submits the form, the component initiate signup with Cognito. If you encounter an error, try checking the following places for more information to help you debug:

  • the browser’s JS console
  • CloudWatch logs for the PreSignup Lambda function
  • Consider adding support to run the standalone shell for react-dev-tools

The example app requires the user to close the app, then relaunch it. That’s because the successful signup request creates the user record in Cognito, but it does not grant the user the tokens it will need to access the app’s backend later. The app requires a Load request from BigCommerce to initiate the custom auth flow, and grant the tokens. The Load request is triggered when the user clicks the app’s icon in their BigCommerce control panel.

Load Callback

The custom user pool authentication flow for Cognito must process the signed payload BigCommerce delivers when a user triggers the Load Callback request by clicking the app’s icon in the Apps menu. The custom authentication flow consists of three Lambda triggers:

  • Define Auth Challenge — Tells Cognito to use CUSTOM_CHALLENGE, issue tokens if the challenge was successful, or reject the auth attempt if the challenge was failed
  • Create Auth Challenge — Tells Cognito the answer is always the BigCommerce client secret
  • Verify Auth Challenge Response — Verifies the signed payload returned by the React Client’s response to the auth challenge with the BigCommerce client secret from the Create Auth Challenge Lambda’s response.
Custom Cognito auth flow sequence diagram — Note: there is a missing participant between Create Auth Challenge and Secrets Manager. Create Auth Challenge invokes the retrieveSecret Lambda to retrieve the Secret from Secrets Manager.

Define Auth Challenge Lambda Function

The Define Auth Challenge Lambda receives an event from Cognito, and based upon the session history in the event it will either direct Cognito which type of challenge to use next, or grant or deny the sign in attempt. Example:

check out the CognitoStoresDefineAuthChallenge code here

Create Auth Challenge Lambda Function

The Create Auth Challenge Lambda is responsible for generating challenge parameters, like the image to for a captcha, and providing the challenge answer. BigCommerce authenticates by sending a signed payload, so the challenge for the React Client to answer correctly is supplying a payload signed by the BigCommerce client secret. In essence, the secret is the challenge answer, so the BigCommerce Client Secret is the answer the Create Auth Challenge Lambda must supply.

1. Grant Access to retrieveSecret Lambda
The Lambda’s IAM policy must include access to the retrieveSecret Lambda, so it can supply the challenge answer. Just like the PreSignup Lambda, you can use Amplify CLI to make the appropriate changes.

$ amplify function update
Using service: Lambda, provided by: awscloudformation
? Please select the Lambda Function you would want to update CognitoStoresCreateAuthChallenge
? Do you want to update permissions granted to this Lambda function to perform on other resources in your project? Yes
? Select the category function
? Function has 5 resources in this project. Select the one you would like your Lambda to access retrieveSecret
? Select the operations you want to permit for retrieveSecret read
You can access the following resource attributes as environment variables from your Lambda function
var environment = process.env.ENV
var region = process.env.REGION
var functionRetrieveSecretName = process.env.FUNCTION_RETRIEVESECRET_NAME
? Do you want to edit the local lambda function now? No
Successfully updated resource

2. Write the Lambda Function

check out the CognitoStoresCreateAuthChallenge code here

Verify Auth Challenge Response Lambda

The Verify Auth Challenge Response Lambda function accepts the challenge answer from the Create Auth Challenge Lambda, and the challenge response from the React client, and returns an object that indicates if the challenge response was correct.

check out the CognitoStoresVerifyAuthChallengeResponse Lambda code here

React Client: Add Load Callback Route

Route
The example app uses react-router-dom to create a route to /load in src/App.js. You may use any router you choose, as long as the resulting route matches the Load Callback URL you supplied to BigCommerce.

check out the routing in App.js here

Load Component
The Load component is rendered by the route in App.js. It is responsible for:

  • initiating the custom auth flow
  • responding to the custom auth challenge with the signed payload
  • render the app’s UI if the auth flow is successful, or display an error otherwise
Load component —Visually,  it is just a progress spinner
Load component — Visually, it is just a progress spinner

When the component is mounted, it uses the signIn() method of the Auth class from aws-amplify to initiate the custom auth flow. Upon successful authorization, the component redirects to a new route that renders the app’s UI.

check out the Load Component code here

API

The React client uses AWS API Gateway to access backend resources. The API Gateway is responsible for:

  1. Routing requests to the appropriate resources
  2. Authenticating requests with Cognito

This first API provides an endpoint that supports a GET request to retrieve information about the store. After Cognito authenticates the identity and access tokens supplied in the request, the endpoint routes to a Lambda function running aws-serverless-express that…

  • …retrieves the user name (BigCommerce store hash) that corresponds to the Cognito user ID from Cognito…
  • …retrieves the API token that corresponds tot eh BigCommerce store hash from DynamoDB
  • …then invokes another Lambda function that acts as the BigCommerce API client.

Potential area of improvement:
A while after I designed this flow, I reconsidered using the Cognito user ID as the primary key in Dynamo DB. Doing so would prevent the need to query Cognito for the username, since Cognito has already verified that the user ID is authentic and the user is authorized to access the BigCommerce API token. But at that point, the app had already been written.

At some point I may revisit this idea since it would have other benefits as well. For instance, Amplify CLI is able to generate a scaffolding for an API to perform CRUD operations on a DynamoDB table. It comes with a built-in setting that, when enabled, uses the Cognito user ID supplied in the identity token as the primary key in the table. That saves a bit of effort writing DB queries.

The BigCommerce API client has no dependencies on other storage or compute resources, so it expects to be supplied with the necessary credentials and parameters to make the request. That means the Lambda that invokes the BigCommerce API client is responsible for supplying the store hash from Cognito, and the store’s API token from DynamoDB.

BigCommerce API Client Lambda Function

The BigCommerce API client Lambda function is responsible for:

  • Validating the parameters supplied in the event
  • Making requests to the BigCommerce API
  • Returning responses from the BigCommmerce API

Start by creating the local resource with AWS CLI

$ amplify function add
Using service: Lambda, provided by: awscloudformation
? Provide a friendly name for your resource to be used as a label for this category in the project: bcApiClient
? Provide the AWS Lambda function name: bcApiClient
? Choose the function template that you want to use: Hello world function
? Do you want to access other resources created in this project from your Lambda function? No
? Do you want to edit the local lambda function now? No
Successfully added resource bcApiClient locally.

The Lambda function needs access to the BigCommerce Client ID to send requests to the BigCommerce API. The example supplies the Client ID as an environment variable.

Modify the CloudFormation Template

1. parameters.json
Create a file at amplify/backend/function/bcApiClient/parameters.json with the following content:

Replace the values:

  • BCCLIENTID: Your BigCommerce Client ID

2. CloudFormation Template Parameters
In the amplify/backend/function/bcApiClient/bcApiClient-cloudformation-template.json CloudFormation template, add references to the Parameters object for the values in parameters.json:

3. Environment Variables
In amplify/backend/function/bcApiClient/bcApiClient-cloudformation-template.json add this to Resources.LambdaFunction.Environment.Variables

4. Write the Lambda Function

The BigCommerce API client accepts parameters for store hash, BigCommerce API token, method, request body, URL parameters, and query parameters. You could write the Lambda to accept any event schema you choose, or consider using the node-bigcommerce API client built by Conversio, but that is beyond the scope of this article.

The method describes both the BigCommerce endpoint, and the request method. Examples:

The example app only implements the GET_STORE method. The webhooks methods are left over from the original Webhooks Manager project, and can serve as a blueprint for creating similar CRUD methods for other endpoints.

check out the bcApiClient code here

API Gateway and Lambda Function

The API is composed of an API Gateway, and a Serverless Express Lambda. Authenticated users will send requests from the React client to the API Gateway. The API Gateway integrates with Cognito to reject unauthorized requests. It routes authorized requests to the Serverless Express Lambda.

The example includes a read-only route at /store-information, as well as a /webhooks route with full CRUD support. The example only implements the /store-information route at the API or in the React client. The webhooks routes are left over from the original Webhooks Manager project, and can serve as a blueprint for creating similar CRUD routes for other endpoints.

The API gateway is responsible for:

  • Defining routes
  • Authenticating Identity and Access Tokens with Cognito
  • Routing authenticated requests to the Lambda function

The Lambda function is responsible for:

  • Defining route handlers
  • Validating request parameters
  • Retrieving and decoding or decrypting the BigCommerce API Token from DynamoDB
  • Invoking the bcApiClient Lambda function
  • Processing responses from the bcApiClient Lambda function
  • Responding to the React Client’s request

Create the API Gateway and Lambda locally with Amplify CLI

$ amplify api add
? Please select from one of the below mentioned services REST
? Provide a friendly name for your resource to be used as a label for this category in the project: bcApi
? Provide a path (e.g., /items) /store-information
? Choose a Lambda source Create a new Lambda function
? Provide a friendly name for your resource to be used as a label for this category in the project: bcApiLambda
? Provide the AWS Lambda function name: bcApiLambda
? Choose the function template that you want to use: Serverless express function (Integration with Amazon API Gateway)
? Do you want to access other resources created in this project from your Lambda function? Yes
? Select the category storage, function, auth
Storage category has a resource called DynamoStores
? Select the operations you want to permit for DynamoStores read
? Function has 5 resources in this project. Select the one you would like your Lambda to access bcApiClient
? Select the operations you want to permit for bcApiClient read
Auth category has a resource called CognitoStores
? Select the operations you want to permit for CognitoStores read
You can access the following resource attributes as environment variables from your Lambda function
var environment = process.env.ENV
var region = process.env.REGION
var storageDynamoStoresName = process.env.STORAGE_DYNAMOSTORES_NAME
var storageDynamoStoresArn = process.env.STORAGE_DYNAMOSTORES_ARN
var functionBcApiClientName = process.env.FUNCTION_BCAPICLIENT_NAME
var authCognitoStoresUserPoolId = process.env.AUTH_COGNITOSTORES_USERPOOLID
? Do you want to edit the local lambda function now? No
Succesfully added the Lambda function locally
? Restrict API access Yes
? Who should have access? Authenticated users only
? What kind of access do you want for Authenticated users? read
? Do you want to add another path? No
Successfully added resource bcApi locally

The CLI will create a scaffolding where you can implement your routes and route handlers in amplify/backend/function/bcApiLambda/src. The example defines all routes and handlers in the app.js file in that directory.

While Cognito supplies tokens that identify the user by user ID, the username (store hash) is required to query the table in DynamoDB to retrieve the store’s API token. So, the Lambda function must request the corresponding username from Cognito, before querying DynamoDB for the BigCommerce access token.

The aws-serverless-express Lambda function will immediately look familiar to anyone who’s used Express. One useful addition is the req.apiGateway object, which you can use to access the event object. This example retrieves the user ID from the identity supplied by Cognito in req.apiGateway.event.requestContext.identity.cognitoAuthenticationProvider as described here by AlexThomas90210.

check out bcApiLambda the code here

UI

Components

The example app’s UI renders a Dashboard component at a /dashboard route in App.js. The dashboard is composed of a Navigation component, and a route to /dashboard/store-information, which in turn renders the StoreInformation component. The idea is that additional routes and components can be added to the Dashboard as needed.

The Dashboard and Navigation components are not particularly remarkable. The StoreInformation component, on the other hand, implements the get() method of the API class from aws-amplify to retrieve data from the BigCommerce API, and render it in the React client.

Check out the StoreInformation code here

Testing

Deploy the Backend
Now is a good time to deploy your new local resources to the cloud and test the app. To deploy the resource, run the following from the root directory of the project:

$ amplify serve

When you are prompted to create the resources in the cloud, answer Y .

Test the App

1. Log into the BigCommerce store

2. Click Apps, then click the App’s icon
Note: If the app’s icon is not present in the Apps menu, make sure the app is installed.

Note: If the app’s icon is not present in the Apps menu, make sure the app is installed.

3. Confirm the UI is rendered

4. Click the Get Store Information button

3. Confirm the UI is rendered
4. Click the Get Store Information button

At this point, the React client should use the API.get() method to invoke the app’s API with the identity and access tokens supplied by Cognito. If you receive an error from the API, check the CloudWatch logs for the bcApiClient and bcApiLambda functions.

Uninstall Callback

Hooray, the app does a thing! That means we’re almost finished, right? Almost right There’s still the Uninstall Callback to implement. When a BigCommerce user with store owner permissions uninstalls the app, BigCommerce invalidates the app’s API tokens. But, the user can reinstall the app at any point. If the app does not remove the user from the Cognito user pool, then the BigCommerce user will receive an error if they attempt to reinstall the app. That could probably be solved in the Auth Callback flow, but the example does so by removing the user from Cognito when BigCommerce sends an Uninstall Callback request.

The example uses AWS Gateway to route the Uninstall Callback request to a Lambda function. BigCommerce will not provide Cognito tokens with the request. Instead, the request includes a signed payload. The app verifies the signature on the payload with the BigCommerce Client Secret to verify authenticity. That means the app’s uninstall API must not enforce authentication with Cognito like the app’s first API. That also means that the Lambda must retrieve the BigCommerce Client Secret with the retrieveSecret Lambda.

Create the Uninstall API and Lambda function with Amplify CLI:

$ amplify api add
? Please select from one of the below mentioned services REST
? Provide a friendly name for your resource to be used as a label for this category in the project: uninstallApi
? Provide a path (e.g., /items) /uninstall
? Choose a Lambda source Create a new Lambda function
? Provide a friendly name for your resource to be used as a label for this category in the project: uninstallLambda
? Provide the AWS Lambda function name: uninstallLambda
? Choose the function template that you want to use: Serverless express function (Integration with Amazon API Gateway)
? Do you want to access other resources created in this project from your Lambda function? Yes
? Select the category storage, function, auth
Storage category has a resource called DynamoStores
? Function has 8 resources in this project. Select the one you would like your Lambda to access retrieveSecret
? Select the operations you want to permit for retrieveSecret read
Storage category has a resource called DynamoStores
? Select the operations you want to permit for DynamoStores delete
Auth category has a resource called CognitoStores
? Select the operations you want to permit for CognitoStores delete
You can access the following resource attributes as environment variables from your Lambda function
var environment = process.env.ENV
var region = process.env.REGION
var storageDynamoStoresName = process.env.STORAGE_DYNAMOSTORES_NAME
var storageDynamoStoresArn = process.env.STORAGE_DYNAMOSTORES_ARN
var functionRetrieveSecretName = process.env.FUNCTION_RETRIEVESECRET_NAME
var authCognitoStoresUserPoolId = process.env.AUTH_COGNITOSTORES_USERPOOLID
? Do you want to edit the local lambda function now? No
Succesfully added the Lambda function locally
? Restrict API access No
? Do you want to add another path? No
Successfully added resource uninstallApi locally

Amplify will create a file at amplify/backend/function/uninstallLambda/src/app.js where you can implement the route handler for the /uninstall path on the API.

check out the uninstallLambda code here

Testing

Deploy the Backend
Now is a good time to deploy your new local resources to the cloud and test the app. To deploy the resource, run the following from the root directory of the project:

$ amplify serve

When you are prompted to create the resources in the cloud, answer Y .

Test the App

1. Retrieve the Uninstall API Endpoint URL

You can retrieve the API’s Invoke URL from the AWS console. Alternatively, you can find it in src/aws-exports.js in your project folder.

2. Provide the Uninstall Callback URL to BigCommerce

2.1 Log in at https://devtools.bigcommerce.com and click the Edit App button.

2.2 On the Technical tab, put the uninstall API’s Invoke URL

Note: If you retrieved the URL from aws-exports.js, you will need to append the path: /uninstall

2.3 Click Update and Close

3. Trigger an Uninstall Callback

3.1 Log in to your store, and navigate to Apps > My Apps

3.2 Click the Uninstall link for your app

At this point, BigCommerce will send an uninstall request to the URL you supplied. You can use AWS console to check the user pool in Cognito and the table in DynamoDB to ensure the store’s user and BigCommerce API Token have been deleted. If they aren’t deleted after uninstalling the app, check the CloudWatch logs for uninstall Lambda. If there are no log streams for the uninstall Lambda, check that the uninstall API’s Invoke URL matches the Uninstall Callback URL provided to BigCommerce.

Hosting

To move app from your local development environment to a staging environment, Amplify CLI can run the app’s build commands and upload the artifacts to S3 hosting with HTTPS termination and a CDN provided by CloudFront. At that point, the Auth and Load Callback URLs that were previously on localhost must be updated with the new CloudFront URLs.

Soonᵀᴹ I hope to publish another article exploring the integration between GitHub and Amplify Console to implement CI/CD from the local environment, to staging and production utilizing three separate BigCommerce apps and Amplify environments. After configuring the environments and pipelines, deployments become nearly trivial.

S3 and Cloudfront

This example will use an S3 Bucket to host the React client, and a Cloudfront distribution to provide HTTPS and a CDN. For a production application, consider using Route 53 to route a domain to the Cloudfront distribution so that you can more easily swap it with another distribution without updating the callback URLs at https://devtools.bigcommerce.com.

Use Amplify CLI to add hosting, and publish the app:

$ amplify hosting add
? Select the environment setup: PROD (S3 with CloudFront using HTTPS)
? hosting bucket name react-client
? index doc for the website index.html
? error doc for the website index.html
You can now publish your app using the following command:
Command: amplify publish
$ amplify publish

When you are prompted to create the resources in the cloud, answer Y .

Amplify will create a CloudFront distribution with HTTPS URLs for your S3 bucket. Then, it runs the React client’s build command, and uploads the artifacts to an S3 Bucket. Finally, the CLI will provide you with a URL where you can access the React Client.

Update Presignup Cognito Environment Variable

You will need to update BigCommerce Dev Tools with the new callback URLs, but remember that the PreSignup Lambda has an environment variable for the app’s Auth Callback URL. So, before updating BigCommerce, update amplify/backend/function/CognitoStoresPreSignup/parameters.json to use the React Client’s CDN endpoint instead of the local server.

Update your backend resources with Amplify CLI:

$ amplify push

When you are prompted to create the resources in the cloud, answer Y .

Update BigCommerce Auth and Load Callback URLs

Next, you need to update Auth and Load Callback URLs in BigCommerce to use the CloudFront distribution, rather than the local development server.

  1. Copy the React Client’s URL from the CLI, or src/aws-exports.js where you can find it as the aws_content_delivery_url.
  2. On the Technical tab, replace the local server’s Auth and Load Callback URLs with the React Client’s CDN URL.
  3. Click Update and Close

Test the App

At this point, your app should be ready for end to end testing. Try installing the app, using it to send requests to the BigCommerce API, and uninstalling the app.

What’s Next?

It works, it was (very nearly) free to develop, and I think it’s pretty cool. I guess that’s a success? Hopefully this helped you, but there’s so much more to do. There’s the signup form to gather additional information that needs building. You can add new tabs to the app for your own utilities, connect your app’s repo to Amplify console to automate your app’s deployment, or use this as a launchpad to integrate BigCommerce with a new system.

For me, I’ve been working with the Amplify Console to automate deployments from my private repo to staging and production environments, added some additional modules like a Customer Login API and JWT decoder utility, and integrated my original Webhooks Manager utility. I hope to write more articles like this one expanding on the sample app. Thanks for reading!

--

--