AWS Serverless Architectural Patterns and Best Practices
In this article, we are going to discuss about How we can use AWS Serverless Architectural Patterns and Best Practices when developing Serverless E-Commerce application.
At the end of the article we will be design the Reference Architecture above which is a Real-world Serverless E-commerce application.
Step by Step Design AWS Architectures w/ Course
I have just published a new course — AWS Serverless Microservices with Patterns & Best Practices.
In this course, we’re going to learn how to Design and Develop AWS Serverless Event-driven Microservices with using AWS Lambda, AWS DynamoDB, AWS API Gateway, AWS EventBridge, AWS SQS, AWS CDK for IaC — Infrastructure as Code tool and AWS CloudWatch for monitoring.
Source Code
Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.
Serverless + CDK Automation + Integration Patterns = AWSome!
I would like to start with quote from Gregor Hophe;
- Serverless + CDK Automation + Integration Patterns = AWSome!
This will be the manifesto of our architecture when designing Serverless Event-driven E-commerce application. We will follow these manifesto every place of our architecture and the Serverless E-commerce application.
First one is Serverless; we have learned that AWS Serverless will be our application development framework.
Second one is AWS CDK; AWS CDK is IaC tool that we will develop whole infrastructure with typescript coding, we will see all detail in the upcoming sections, but the idea is here we will follow the IaC best practices.
And last one is Integration Patterns; that we will follow Queue-Chaning, Publish-Subscribe and Fan-out design patterns about integrations in this article.
So when we combine all of these keywords, it becomes awsome microservices !
Serverless Architectural Patterns
Now we can talk about which Serverless design patterns we will use. And we will focus on Application Integration Patterns for microservices.
Serverless architecture is a way to build and run applications and services without having to manage infrastructure. The approach helps teams focus on real business value and forget about Infrastructure management. Serverless-based architecture has many other advantages and you can find blogs covering them.
RESTful Microservices Pattern
This is Synchronous Communication for RESTful Microservices. This is simple web service is the most standard use-case for AWS Lambda as a backend service. Clients typically communicate via REST APIs with the backend services. Synchronous commands are request/response.
In synchronous model, the client makes a request and waits for a response. We will developed RESTful Microservices with API Gateway — Lambda Functions and DynamoDB.
For Public APIs, API Gateway is exposing lambda functions via HTTPs. API Gateway can handle authorization, authentication, routing, and versioning. For internal APIs, clients invoke lambda functions directly from within the client app using AWS SDK.
We called this; RESTful Microservices;
- REST API and CRUD endpoints with using AWS Lambda, API Gateway
- Data persistence with using AWS DynamoDB
AWS API Gateway -for- Restful API-Driven Development for performing CRUD operations amongst lambda and DynamoDB.
Fan-Out & Message Filtering with Publish/Subscribe Pattern
This is an asynchronous design, This communication is basically for performing one-to-many and publish/subscribe mechanisms that has multiple receivers.
The client service publish a message and it consumes from several microservices which's are subscribing this message on the message broker system.
The client sends a request and may get an acknowledgement that the event was received, but it doesn’t get a response that includes the results of the request.
It decouples messaging between services. Asynchronous messaging is the foundation for most service integrations. It allows building loosely-coupled architectures that overcome the limits of remote service communication, like latency and unreliability. Asynchronous messaging enables services to announce events to multiple interested consumers without coupling.
In Publisher-Subscriber services publish events through a channel as messages. Multiple interested consumers listen to the events by subscribing to these channels.
Mostly this Asynchronous communication is using in event-driven architectures. In Asynchronous event-driven communication, microservice publishes an event when something happens. Example use case can be like a price change in a product microservice. Price Changed event can subscribed from SC microservice in order to update basket price asynchronously.
Publish/Subscribe Pattern with Amazon EventBridge
We will use Amazon EventBridge -for- Event-Driven asynchronous Communication between Microservices.
In Amazon EventBridge we have
- Event Sources
- Event Buses
- Rules
- Targets
We will develop EventBridge with AWS Lambda Basket microservices which publish the event in asynchronous way and this will consume by Ordering queue.
Topic-Queue Chaining & Load Balancing Pattern
Using a queue that acts as a buffer between the service from which it was called from asynchronous invocations. By this way we can avoid loss data if the service to fail or the task to time out. This can help minimize the impact of peaks in demand on availability and responsiveness for the consumer microservice.
If we look at the Topic-Queue Chaining pattern, — you can see picture below;
There are 3 subscriber backend services;
- Customer Notification Services which’s interested in getting notified from publisher microservices.
- Customer Accounting Services
- Extraordinary ride Service
So if one of these services can be down or getting exception or taken offline for maintenance, then the events will be loses, disappeared and can’t process after the subscriber service is up and running.
Because publisher service publish messages with fire & forget method, don’t aware of which services consume this events and how to process them. So if we subscribe from any ephemeral service like microservice or lambda functions, those services can potentially lose or miss topic messages from coming the EventBus.
A good pattern to apply here is topic-queue-chaining. That means that you add a queue, in our case an Amazon SQS queue, between the Amazon EventBridge and each of the subscriber services.
In our Serverless microservice architecture, we put Amazon SQS between Amazon EvenBridge and Ordering microservices. Because checkout basket event publish from Basket microservices and its subscribe from Ordering microservice.
So if we store this event messages into SQS queue with durable and persistent manner, no message will get lost should a subscriber process run into problems for many hours or days, or has exceptions or crashes.
But there is more advantage to put SQS in front of subscriber service. With putting Amazon SQS queue in front of subscriber service, we can take advantage of the fact that a queue can act as a buffering load balancer.
Due to the nature of each queue message being potentially delivered to one of many consumer processes, you can easily expand your subscriber services and the message load will be distributed over existing consumer processes.
Also, due to messages are saved in the queue, a scaling event won’t cause you to lose messages, for example, if you need to wait until an additional consumer operation becomes operational.
So We will use;
- AWS SQS -for- Decouple Microservices and processing events asynchronously using queues
We will develop AWS SQS with Lambda Ordering microservices which consumes by polling the event from the Ordering queue.
We use Event Source Mapping for Polling event messages in Ordering Lambda microservices. In a polling event, consumers poll the producer for messages in batches and then process the batch before returning for a new batch of records.
Before we applied the Serverless Patterns, let’s design our enterprise E-Commerce application with fully Serverless Event-driven Microservices Architecture.
AWS Serverless Microservices for Ecommerce Application Architecture
Here you can find the main overall Serverless Architecture for our application. This is the big picture of what we are going to develop together for AWS Serverless Event-driven E-commerce Microservices application that is Step by Step Implementation together.
We will be following the Reference architecture above which is a real-world Serverless E-commerce application and it includes;
- REST API and CRUD endpoints with using AWS Lambda, API Gateway
- Data persistence with using AWS DynamoDB
- Decouple microservices with events using Amazon EventBridge
- Message Queues for cross-service communication using AWS SQS
- Cloud stack development with IaC using AWS CloudFormation CDK
Serverless Architectural Patterns in Ecommerce Microservices Application
Lets see how we can use these Serverless pattern into our e-commerce application.
RESTful Microservices Pattern
This is Synchronous Communication for RESTful Microservices. This is simple web service is the most standard use-case for AWS Lambda as a backend service.
We called this = RESTful Microservices
- REST API and CRUD endpoints (AWS Lambda, API Gateway)
- Data persistence (AWS DynamoDB)
Fan-Out & Message Filtering with Publish/Subscribe Pattern
This is an asynchronous design, The Basket microservice publish a basketCheckout event message and it consumes from Ordering microservices. It decouples Messaging between Basket and Ordering microservices with using Amazon EventBridge as a Eventbus.
Topic-Queue Chaining & Load Balancing Pattern
We will use AWS SQS -for- Decouple Microservices and processing events asynchronously using queues. We will develop AWS SQS with Lambda Ordering microservices which consumes by polling the event from the Ordering queue.
Storing this event messages into SQS queue with durable and persistent manner, no message will get lost should a subscriber process run into problems for many hours or days, or has exceptions or crashes.
As you can see that we have seen Serverless Architectural Patterns and Best Practices that we use when developing our Serverless e-commerce application.
But lets finish with our main manifesto of architecture with quote from Gregor Hophe;
- Serverless + CDK Automation + Integration Patterns = AWSome!
Step by Step Design AWS Architectures w/ Course
I have just published a new course — AWS Serverless Microservices with Patterns & Best Practices.
In this course, we’re going to learn how to Design and Develop AWS Serverless Event-driven Microservices with using AWS Lambda, AWS DynamoDB, AWS API Gateway, AWS EventBridge, AWS SQS, AWS CDK for IaC — Infrastructure as Code tool and AWS CloudWatch for monitoring.
Source Code
Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.
