Going Serverless

Before diving into this hot topic, some clarifications are needed. Serverless does not mean your application is without a server. It means you, the developer, does not have to worry about the web framework (Play, Node, Vertx ..etc) or the infrastructure setup required. Only cloud services are used to manage the applications and you just have to worry about writing the actual business logic.

Introduction:

The serverless architecture consists of two parts: Trigger and Lambda. The Trigger could come from many sources within the AWS Ecosystem. It could be a data persist in DynamoDB, file update in S3, API Gateway call, SNS message, etc. The Lambda, essentially a remote procedure in AWS, would be invoked by the trigger and carry out some functionalities.

A simple example of a severless application could be a thumbnail creation process for a music serving company. The steps would include:

  • Employee upload a song with album art to a designated S3 bucket (Trigger)
  • Lambda is triggered to resize the image to a thumbnail and persist it to DynamoDB (Lambda)

A Real-life Example:

My use case of the serverless micro-service is to relay a set of sensor data collected from sensor boards to a designated RabbitMQ repository. Serverless architecture makes sense for the task because it is lightweight and has minimal infrastructure setup. I will dive into the three main parts required to build this application. They are:

  • Cloudformation template
  • Lambda
  • Provisioning Script

Cloudformation Template:

The cloudformation template is a configuration file for defining AWS resources. It is primarily used by the AWS platform for standing up a stack of technologies for a service/application. In my case, I need an API Gateway that allows the sensor boards to post data and a Lambda containing logic for connecting to RabbitMQ and pass data to it. With AWS’ latest SAM (Server-less Application Model) Cloudformation format, it is easier than ever to standup the components that I needed. Here is my template:

See https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md

There are two main components defined in the template, SensorApi and DataPublish.

SensorApi is the trigger in the service. There are a few key attributes in its definition:

  • Type — tells the interpreter that its an API endpoint
  • DependsOn — indicates that the DataPublish Lambda is a dependency
  • DefinitionBody — contains the Swagger template of the API. It simply defines a single post endpoint that consumes the SensorData object. By providing the x-amazon-apigateway-integration field, the template informs cloudformation that the Lambda is the handler of the endpoint.

DataPublish is the Lambda. The key attributes in this component are:

  • Handler — the value index.handleEvent means the function handleEvent in index.js is the entry function of the Lambda
  • Runtime — Lambda runtime (NodeJs in this case)
  • CodeUri — local path to code source (src folder in the project root)
  • Environment — Lambda environment variables
  • Events — triggers that would invoke this lambda

This template sets up the infrastructure that I needed for my service. With 61 lines of configuration code I have:

  • Defined an API Gateway, a Gateway Path and Resource
  • Defined a fully configured Lambda with environment data
  • Established the connection between components
  • Setup all the necessary IAM roles required to execute the AWS components

Lambda:

As the template specified, I have to create a Lambda called index.js file with an entry method called handleEvent under the src folder. I will not post the actual code since it is irrelevant to the learning of our topic. Here is a sample of what it looks like:

src/index.js

exports.handleEvent = (event, context, callback) => {
  //…
// 1) parse data passed in through the 'event' object
// 2) create a connection to RabbitMQ cluster
// 3) send over data
//…
}

The main thing to remember is that the Lambda source code contains the business logic for the service. More information related to AWS Lambda can be found here: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html.

Provisioning Script:

When you complete the Lambda the final step is running the cloudformation template and provision the components. Here is the script:

text in brackets are variable names, replace without brackets

The script does three things:

  • Create an S3 bucket for storing the lambda in cloud
  • Interpret the input template (my cloudformation template) and generate an output template that inserts actual S3 url in place of the CodeUri. The reason this step is needed is that AWS cannot access the local Lambda source files. Therefore, the code must be uploaded to a S3 bucket first.
  • Execute the cloudformation script and provision the components under the stack-name

Running the script would create the entire infrastructure without any manual intervention. Aside from the actual logic of parsing sensor data and passing them to RabbitMQ, this micro-service is generally complete.

Pros and Cons:

Let us first dive into the benefits of going serverless.

  • Simplicity — the main reason for going serverless is to reduce amount of infrastructure setup and minimize the work required to research web frameworks and implement them. Even though there might be some initial learning required to understand AWS cloudformation syntax and commands, it is a one time investment that goes a long way. It is ubiquitous in any web projects that uses AWS cloud as its infrastructure (which is almost every web project).
  • Flexibility — The beauty of the cloudformation template is that it allows you to specify the triggers, lambda and the relationship between them. Unlike an unwieldy package like a WAR or ZIP file, the triggers and lambas are modular components that could be rearranged. You can mix and match different triggers and lambdas. For example, if I want to change the API to a SNS (message queue) trigger down the line, all I have to do is to update the cloudformation template to spin up an SNS component instead of an API endpoint.
  • Reliability — AWS platform is proven and scales extremely well. Since you are simply writing the cloudformation instead of creating the stacks yourself, you don’t have to worry about things like web framework updates that could break your infrastructure. You simply rerun the cloudformation template and the stack would be refreshed.

The technology is not without flaws.

  • Testing — testing locally is impossible. Unlike a web application, you cannot simply download the source code, spin up the application and test it in localhost. Even though you could update the template variables and spin up a local version of the stack in the cloud, it’s not the same. Unit testing is possible since you can write tests for all the lambda code.
  • Code Sharing — The advantage of having an web application is sharing code for common procedures like persisting to database or opening an HTTP connection. Since each lambda is its own individual component, its modularity is also its curse.

Conclusion:

This is a high level introduction to AWS serverless framework and I have purposely left out many details such as using configuration management tool along with cloudformation to set up multiple environments (I will write a follow up post if there is a demand for it). Like any other tools, there is a place and a time for it. Anyways, I hope you find this introduction useful. Happy coding.