Building a serverless application on AWS — Part 1: Create a simple API and ship it to AWS

In my first article series, I want to give you a brief example of how to build a serverless application on AWS. I will provide you everything we discussed in this article on GitHub which allows you to go step by step through the source-code again. I will use AWS CloudFormation to provision this application on AWS (with the benefit of having our Infrastructure as Code).

So what does it mean in detail? In the end of this series, you should be able to build a serverless application on AWS, know the basic usage of AWS CloudFormation/AWS API Gateway/AWS Lambda/AWS SNS and AWS DynamoDB.

Prerequisites

Before we start let’s make sure that you can follow this article series properly. If you answer each question with “Yes” please go ahead. If not, you should do your homework before continuing ;-)

• Do you have an AWS account up and running?
• Do you have the AWS CLI installed and configured?
• Do you have enough permissions for all services mentioned in the next section?
• Do you know what “AWS Lambda” is?
• Do you know what “AWS CloudFormation” is?
• Do you know how to setup an express application in Node.js?

Photo by Evan Dennis on Unsplash

Which AWS services are gonna to be used in this article series?

• AWS CloudFormation
• AWS Lambda
• AWS API Gateway
• AWS DynamoDB
• AWS Identity and Access Management (IAM)
• AWS Simple Notification Service (SNS)
• AWS Simple Storage Service (S3)

Which parts exists in this article series?

• Part 1 (this article): Create a simple API and ship it to AWS
• Part 2: Create a DynamoDB table and store data
• Part 3 (tbd): Validate incoming images and create thumbnails from them
• Part 4 (tbd): Implement basic usage of Event-Sourcing and create SNS event publisher
• Part 5 (tbd): Create a simple CI/CD pipeline

Part 1 — Let’s get started!

Say you have the following API which only prints different messages on each route.

src/api.js

With that, you are already able to run the application on your local machine and/or on a virtual server. Important: while developing you shouldn’t take care how to deliver your application later to your customers. Your main goal while developing should be that you, as a developer, can easily develop and test your application on your local machine.

AWS supports you while achieving this goal with an npm package (aws-serverless-express) they created. It wraps an existing express application to be handled by an AWS Lambda. To use that we need to do some small adjustments in our existing API.

src/api.js (with small adjustments to support AWS Lambda)

To summarize the changes: AWS Lambda is executed differently then an HTTP-Server (e.g. nginx), therefore you don’t need to setup a port to listen to.

So how to tell AWS Lambda to use my express application? As I previously mentioned it’s now time to use the aws-serverless-express package. After running npm install aws-serverless-express create a new file src/api-lambda.js with the following content:

src/api-lambda.js

With this two files we are done — from the application perspective. But how to ship it to AWS now? AWS CloudFormation is the right answer!

infrastructure.yml

First, we need to create all resources which are needed to spin up an AWS Lambda. In this example you are using Node.js 8.10 as your runtime (line 21) with a maximum memory size of 128MB (line 22). As soon as your AWS Lambda takes longer than 5 seconds (line 23) to fulfill the invocation it will fail with an timeout which, in the end, results in an error for your customer(s). Each AWS Lambda requires an execution role attached. In this example we attached a role (line 20) with one policy attached (line 14): AWSLambdaBasicExecutionRole which allows AWS Lambda to write into AWS CloudWatch logs (very helpful for debugging purposes).

How to connect AWS Lambda with the outside world?

At this point AWS API Gateway comes into play. There are two ways to configure it:

1. Create specific paths and methods one by one (Figure 1)
2. Create proxy (Figure 2)

Figure 1

Figure 2

You automatically decided to go with proxy configurations as soon as you decided to use express in your application. The proxy configuration enables you to receive and handle each request as you would do on your local machine or/and on a virtual server. Each request received by AWS API Gateway gets mapped (with help of aws-serverless-express) to the internal request structure of express. The same gets applied for the response just vice versa: the response gets mapped from the internal express structure to the structure AWS API Gateway expects.

infrastructure.yml

Above you see all resources which are needed to create an AWS API Gateway.

• API (line 1–4)
• API Method (line 6–30)
• API Deployment (line 32–37)
• API Stage (line 39–44)

But one final piece is missing to connect AWS API Gateway with your AWS Lambda successfully:

infrastructure.yml

AWS API Gateway is by default not allowed to invoke your AWS Lambda. To tackle it we need one additional resource which allows AWS API Gateway (line 6–7) to invoke (line 4) your AWS Lambda (line 5).

Before interacting with an API most of the libraries are doing a preflight request to check if the request gets accepted by the back-end. To do that they are doing an OPTIONS request against your API which delivers then important headers like Access-Control-Allow-Headers, Access-Control-Allow-Methods, Access-Control-Allow-Orgin, etc. . You should definitely support this method but you don’t really want to pay for that additional call against your API that doesn’t bring your business a real value… The following resource will help you :-)

infrastructure.yml

This resource looks similar as the ApiMethod we defined previously. The only difference is the Integration (line 8–20) — AWS API Gateway offers a Mock type (line 20) which brings the benefit of not invoking your AWS Lambda as soon as an OPTIONS request (line 6) hits the API. AWS API Gateway responds with an 200 status code and an empty application/json body instead (to fulfill the OPTIONS request).

We are now done with our infrastructure.yml you can find the final version of it on GitHub.

How to ship it to AWS?

Since we didn’t define the required Code property for our AWS::Lambda::Function resource we need to pre-process our infrastructure.yml template. To do that we are using the aws cloudformation packagecommand provided by the AWS CLI:

aws cloudformation package --s3-bucket some-s3-bucket --s3-prefix aws-serverless-example --template-file ./infrastructure.yml --output-template-file ./infrastructure-packaged.yml

What does the command do?

Packages the local artifacts (local paths) that your AWS CloudFormation template references. The command uploads local artifacts, such as source code for an AWS Lambda function or a Swagger file for an AWS API Gateway REST API, to an S3 bucket. The command returns a copy of your template, replacing references to local artifacts with the S3 location where the command uploaded the artifacts.

[…]

If you specify a file, the command directly uploads it to the S3 bucket. If you specify a folder, the command zips the folder and then uploads the .zip file.

[…]

As an output of the command you will get the infrastructure-packaged.yml template which is ready to get uploaded in AWS CloudFormation. Also here there are two ways to do that:

1. Login in AWS console, go to the CloudFormation console, click “Create Stack”, choose “infrastructure-packaged.yml” as the template file and follow the instructions inside the wizard
2. Use the AWS CLI

For now we are going to use the AWS CLI but I highly recommend everyone who is new to AWS (especially to AWS CloudFormation) to try the manual way first to know which capabilities exists to configure AWS CloudFormation further.

The AWS CLI commands to create and update the stack look like this:

// create stack
aws cloudformation create-stack --stack-name AWS-SERVERLESS-EXAMPLE --template-body file:///$PWD/infrastructure-packaged.yml --capabilities CAPABILITY_IAM --region eu-west-1
// update stack
aws cloudformation update-stack --stack-name AWS-SERVERLESS-EXAMPLE --template-body file:///$PWD/infrastructure-packaged.yml --capabilities CAPABILITY_IAM --region eu-west-1

The final commands to manage your AWS CloudFormation stack are:

// 1. pre-process AWS CloudFormation template
aws cloudformation package --s3-bucket some-s3-bucket --s3-prefix aws-serverless-example --template-file ./infrastructure.yml --output-template-file ./infrastructure-packaged.yml
// 2. create stack
aws cloudformation create-stack --stack-name AWS-SERVERLESS-EXAMPLE --template-body file:///$PWD/infrastructure-packaged.yml --capabilities CAPABILITY_IAM --region eu-west-1
// 3. update stack (as soon as the stack is already created)
aws cloudformation update-stack --stack-name AWS-SERVERLESS-EXAMPLE --template-body file:///$PWD/infrastructure-packaged.yml --capabilities CAPABILITY_IAM --region eu-west-1

Please keep in mind to create an AWS S3 bucket in the same region as you want to create the AWS CloudFormation stack. In my example I used "some-s3-bucket" but you should change it to your bucket name.

As soon as you ran the commands the AWS CloudFormation stack is created and you should see underneath “Outputs” the endpoint of your API.

That’s basically it. Feel free to check the GitHub repository and see how it evolves through this article series… Hope I helped you a little bit in the big -serverless- world. Stay tuned!

-Maik