Microservices with Serverless & AWS Lambda Layers — A Walkthrough

Markus Wagner
Dec 10, 2019 · 5 min read
Photo by dhe haivan on Unsplash

Introduction

Since Microservices are becoming more popular, the technology used to create these architectures is becoming more important as well. We want to keep these architectures both scalable and elastic. So, making use of a cloud service like AWS Lambda is gaining popularity as well!

For efficient development & deployments, our team at Zoi uses Node.JS 10 and the Serverless framework for our projects.

I would describe a Microservice as a small service that works in a set and shares business logic. The architecture must keep the system very flexible, so that, for instance, a single service can be updated while the rest of the system keeps running. That means the service runs self-contained and blue-green deployments are more easier to handle.

Dependencies should also be shared in the services, which simplifies the development of these systems. This article describes how we can make use of the powerful Serverless Tool combined with AWS Lambda Layers and API Gateway to create mid-sized to big enterprise-grade microservices-based architectures.

Full Example: https://github.com/markuswagner93/serverless-layers-example

Reason / Up- and Downsides.

Is there a reason, that requires switching to an architecture with Layers in Lambda?

Yes, when your systems grow too big for a single deployment. The CloudFormation template, which will be created by serverless is limited to 200 resources. Each Lambda function with an HTTP endpoint, for example, creates 4 resources in your deployment.

Splitting these services will allow us to be more flexible and also make the structure and error more complex to handle. The main benefit of using a Layer in Lambda is to share dependencies and business-logic between different functions.

The limits of Layers are defined in the AWS Documentation (Cited 28.11.2019):

  • A Lambda can hook up to only 5 Layers at once
  • Like Lambda, it’s not allowed to be bigger than 250MB
  • Deployed per Account

Example

Enough with the theory, let’s show with an example how AWS Lambda Layers works in practice. The steps can also be looked up in the Repository (Branch step_1, step_2 etc.). All the serverless.yml files will be created manually.

Step 1

First, we will create a simple Lambda and a redirect on the API Gateway through a domain. This is the example for a Serverless Hello World Application (Example Branch):

business-layer/serverless.yml (Source: Author)
business-layer/handler.js (Source: Author)

Step 2

After that, we create the Layer that will contain our dependencies. Simply create the folders, and run “npm init” in each folder (Example Branch):

Folder structure
dependencies-layer/package.json (Source: Author)

Here, we can define all the node packages as usual. In this very example, it’s only the popular node.js library “moment” that is used for formatting dates and our business dependencies. It’s possible to install your own dependencies with npm by running “npm install ../business-dependencies” and moment with “npm install moment” in the directory.

We will also create a serverless.yml file in the root of the project that looks like this:

dependencies-layer/serverless.yml (Source: Author)

Here, we define the layer, which needs a name, the path of the layer, and the compatibleRuntimes. You can imagine that the layer works like a docker image. When a layer is hooked onto a Lambda, it will now load all the dependencies for it.

Step 3

Now we need to define the layer in the lambda, we will use a variable (that could be generated by e.g. Terraform) and attach it for multi stage deployments in the Lambda. In this example, we load a json file and use the account_id property (Example Branch):

business-layer/handler.js (Source: Author)
business-layer/serverless.yml (Source: Author)

Since the term “latest” is listed as the version, we will need a plugin called serverless-latest-layer-version, which will enable the program to always use the latest layer version of the dependency layer. That’s it! Now, we can deploy the layer and then the latest version of the layer afterwards. We will run `serverless deploy — stage dev` in the directory of the dependency layer and then again in your lambda to update it with the current version of the Lambda. The full example can be looked up at GitHub.

Outro

Thanks for reading this! I hope you can now see how flexible, extensive and awesome this architecture can be! For example, updating a single api currently takes about 2 minutes, even with only uploading around 3kB of Code. Since you can hook up to 5 Layers on a single Lambda, we are able to split our dependencies in smaller chunks. So, we use only relevant libraries for our specific Lambdas. The complete API takes about 10 Minutes to perform a full update, which will likely be improved as it gets developed further.

zoiwrites

Read about what we do at Zoi

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store