Decompose Lambda Functions — Monolithic, Microservices vs Single-Purposed Functions
When you are developing Serverless applications, How you decide to create Lambda Functions ? What is the best way to design Lambda microservices ? How to organize Lambda functions into separate services ? Monolithic, Microservices vs Single-Purposed Functions ?
When you are developing your Serverless application, it is easy to develop a few simple Lambda functions. But things changes very fast and you keep creating and developing functions and boom ! Congratulations you have hundreds of lambda functions so now what’s next ?
Step by Step Design AWS Architectures w/ Course
I have just published a new course — AWS Serverless Microservices with Patterns & Best Practices.
In this course, we’re going to learn how to Design and Develop AWS Serverless Event-driven Microservices with using AWS Lambda, AWS DynamoDB, AWS API Gateway, AWS EventBridge, AWS SQS, AWS CDK for IaC — Infrastructure as Code tool and AWS CloudWatch for monitoring.
Source Code
Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.
Function Hell
If you don’t follow your lambda functions, most probably you will end up with function hell case. When your Serverless application is growing, it can be really nightmare to manage your functions.
So What is the best way to design Lambda microservices ? How to organize Lambda functions into separate services ? or Should we ok with hundreds of lambda functions ?
Since there is no clear solution of this problem, AWS has offers some approaches that following the best practices when designing larger number of lambda function applications. There is no one-size-fits-all approach, and each alternative has pros and cons that we need to consider.
Before that let’s remember the Lambda function and the purpose and usage of Serverless functions.
What is AWS Lambda ?
AWS Lambda is an event-driven, serverless computing platform that runs code in response to events and automatically manages the computing resources required by that code. You can run code without thinking any servers or underlying services.
AWS Lambda is a compute service that runs custom function code and return response to events. Most AWS services generate events for communicating each other, and most of AWS Services are event sources for Lambda. So AWS Lambda is very good fit with event-driven architectures, because the nature of lambda executions trigger from events. Events can come lots of resources and able to trigger lambda functions.
When a function is triggered by an event, this is called an invocation.
Lambda functions are limited to 15 minutes in duration, but on average, most calls take less than a second across all AWS customers.
So even the simplest Lambda-based application uses at least one event. So that’s why we said, AWS Lambda is very good fit for the Event-Driven Architectures. Because the fuel of AWS Lambda is events.
How does AWS Lambda work?
We said that AWS Lambda fits into the event-driven architectures. So, as you can see the image AWS Lambda triggers by event and executes the function code.
Each Lambda function runs in its own container. You can think every lambda function as a standalone docker containers. When a function is created, Lambda packages it into a new container and then executes that container on a multi-region cloud clusters of servers managed by AWS.
Using AWS Lambda as a Microservice
Before this article, we have explained that;
- We can use AWS as an Application Development Framework,
- also we can use AWS Lambda as a Microservice,
So we can put AWS Lambda as a microservice in our Serverless Projects like E-Commerce application.
The Serverless Microservices Pattern reduces the barrier for the creation of each subsequent microservice. API Gateway even allows for the cloning of existing APIs, and using of Lambda functions in other accounts. Amazon API Gateway provides programmatically generated client SDKs that are programmatically built in a number of popular languages in order to reduce the integration overhead.
You can see the figure above that shows the architecture of a Serverless microservice with Amazon API Gateway and AWS Lambda. So API Gateway APIs proxy to individual microservices backed by Lambda functions. You can see the microservices-1–2–3–4 that proxy by API Gateway and all of microservices handle with AWS Lambda functions.
Starting Lambda-Based Serverless Application
Once you have decided to use AWS Lambda as a main compute serverless service of your project, then we can start thinking to organize our functions.
There are multiple approaches to develop serverless applications but before starting your project, I assume that you are using serverless deployment frameworks such as the AWS Serverless Application Model (SAM) or the Serverless Framework that can make it easier to group common pieces of functionality into smaller services. So that means you have something like a template.yaml file contains all the resources and function definitions needed for an application. And of course, We will follow the best practice and organizing larger lambda projects when developing our lambda functions.
Organize Lambda Functions
But event you follow the best practices, you will face that continuously growing number of lambda functions living independently from each other. It become extremely hard to navigate between function and determine which lambda function related to certain service.
So we should think about how we can decompose Lambda Functions as per the our business requirements on serverless project.
We have 3 ways to organize our AWS Lambda Serverless functions;
- Single-Purposed Functions
- Microservice-Based Functions
- Monolithic Functions
Now let’s elaborate these approaches according to implementation types in the serverless projects.
Single-Purposed Serverless Functions
As the name suggested, this approach follows to implement only specific task based development into serverless functions. A function will do just one thing and the code for it will be as simple as possible. This will becomes of many tiny individual atomic functions into serverless project.
A good example is RESTFUL API development to perform CRUD operations with multiple endpoints and HTTP methods. Let’s assume we are building a backend in Nodejs for managing Products.
The code for the getProduct, createProduct and deleteProduct functions are stored in different AWS Lambda functions of index.js files and exported 3 lambda functions that triggers over the API Gateway.
Following this approach, your team will implement separate 5 functions to manage Order: getProduct, listProducts, createProduct, updateProduct, and deleteProduct.
So if the team will follow this approach and create functions for each single task, in many cases, resulting in hundreds of functions and you will face function hell issue.
Benefits and Drawbacks of Single-Purposed Functions
The benefit is that each function is completely atomic and can have tailored set of operational characteristics, such as ;
- Granular permissions needed for the function. E.g. getProduct will have ready-only permission to access the Product table, while createProduct will have write permission.
- Tailored resource restrictions. Each function can have a separate set of restrictions around memory, CPU, etc.
- Each pull request for minor changes in the individual function can be deployed separately.
So that means, with single-purpose functions, you can assign more granular permissions to each of the functions. As such, if any of the functions are compromised, the attacker would not gain full access to the data. This provides much better security for your application.
Performance can be problem when you follow single-purposed approach. Because you will face the frequency of cold starts. Every time the FaaS platform scales your function to meet increased traffic, you will experience a cold start.
Microservice-Based Serverless Functions
As the name suggested, this approach follows to implement microservices architecture when developing serverless functions. So that means we can follow the microservices decomposition patterns when designing serverless lambda function.
Basically microservice-based approach suggests keeping multiple individual closely related functions together. We can also say that create serverless functions as per Bounded Contexts of our projects by grouping business capabilities.
If we revisit the first simple example from above, instead of creating 5 separate lambda serverless functions for the management of Products, one can create a single lambda function that handle all of these cases.
Also you can see the above example of E-Commerce architecture implemented by microservice-based lambda functions with Product, Basket and Ordering microservices that communicate over serverless event buses.
The microservice pattern is a concept of keeping each of your services modular and lightweight. So for example; In our e-commerce app allows users to get products, add to basket and checkout basket in order to start ordering process.
Implementation of this approach is required to look into the event object which comes from Amazon API Gateway, and develop switch-case statements for the operation based codes on the http-method fields.
You can find example implementation of this approach;
// index.js
// This is the code for management of Products.
//
// Note:
// The code snippet below is just for demo.
// It includes a simple happy-path without error
exports.handler = async function(event, context) {
switch (event.httpMethod) {
case 'GET':
return await getProduct(event);
case 'POST':
return await createProduct(event);
case 'PUT':
return await updateProduct(event);
case 'DELETE':
return await deleteProduct(event);
default:
return await unknownOperation(event.httpMethod)
}
}Note that getProduct will have to further separate into two functions; getProductById if an id parameter was provided and getAllProducts if not. You can check example implementation in here.
In a typical medium size project, this approach keeps the total number of functions in the 10s, vs. 100s. And for a typical scrum dev team, that owns only a limited number of objects, this model provides a good construct to manage each object type separately.
Benefits and Drawbacks of Microservice-Based Functions
Lambda functions are a natural fit for a microservice based architecture. This is due to a few of reasons. Firstly, the performance of Lambda functions is related to the size of the function. Secondly, debugging a Lambda function that deals with a specific event is much easier. Finally, it is just easier to conceptually relate a Lambda function with a single event.
The easiest way to share code between services is by having them all together in a single repository. Even though your services end up dealing with separate portions of your app, they still might need to share some code between them.
But also there are some drawbacks about this approaches;
- Microservices can grow out of control and each added service increases the complexity of your application.
- This also means that you can end up with hundreds of Lambda functions.
- Managing deployments for all these services and functions can get complicated.
Monolithic Serverless Functions
Many serverless applications begin as monolithic applications. This can occur either because a simple application has grown more complex over time, or because developers are following existing development practices. A monolithic application is represented by a single AWS Lambda function performing multiple tasks, and a mono-repo is a single repository containing the entire application logic.
Monoliths work well for the simplest serverless applications that perform single-purpose functions. These are small applications such as cron jobs, data processing tasks, and some asynchronous processes. As those applications evolve into workflows or develop new features, it becomes important to refactor the code into smaller services.
Comparison of Monolithic and Single-Purposed Functions
Here you can find the table that comparison of Monolithic and Single-Purposed functions. With single-purpose functions, you can assign more granular permissions to each of the functions. This provides much better security for your application. But it can negative affect to performance due to face cold start issues more frequently.
AWS Serverless Microservices for Ecommerce Application Architecture
Here you can find the main overall Serverless Architecture for our application. This is the big picture of what we are going to develop together for AWS Serverless Event-driven E-commerce Microservices application that is Step by Step Implementation together.
We will be following the reference architecture above which is a real-world Serverless E-commerce application and it includes;
- REST API and CRUD endpoints with using AWS Lambda, API Gateway
- Data persistence with using AWS DynamoDB
- Decouple microservices with events using AWS EventBridge
- Message Queues for cross-service communication using AWS SQS
- Cloud stack development with IaC using AWS CloudFormation CDK
Step by Step Design AWS Architectures w/ Course
I have just published a new course — AWS Serverless Microservices with Patterns & Best Practices.
In this course, we’re going to learn how to Design and Develop AWS Serverless Event-driven Microservices with using AWS Lambda, AWS DynamoDB, AWS API Gateway, AWS EventBridge, AWS SQS, AWS CDK for IaC — Infrastructure as Code tool and AWS CloudWatch for monitoring.
Source Code
Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.
