Event-Driven Microservice Architecture using Amazon EventBridge, SQS and Lambda

In this article, we are going to develop Event-Driven Microservice Architecture Using Amazon EventBridge, SQS and Lambda.

Event-Driven Microservice Architecture with AWS

By the end of the article, we will develop Hands-on Labs : Event-Driven Microservice Architecture Using Amazon EventBridge, SQS and Lambda
and we will see how to Decouple Microservices with Events using Amazon EventBridge.

I have just published a new course — AWS Lambda & Serverless — Developer Guide with Hands-on Labs.

Event-Driven Microservices Architectures

Basically event-driven microservice architecture is means communicating with microservices via event messages. With the event-driven architectures we can do asynchronous behavior and loosely coupled structures. In example, instead of sending request when data needed, services consume them via events. This will provide to performance increases.

Event-Driven Microservices Architectures

But also there are huge Innovations on the Event-Driven Microservices Architectures like using real-time messaging platforms, stream-processing, event hubs, real-time processing, batch processing, data intelligence and so on.
Let me recap what is Event-Driven Architecture, Basically we use events when implementing our use cases. For example, for our e-commerce application we have a few microservices like: customer, order, payment and products.

And we have several use cases across microservices like that: a customer create orders with some products and, if the payment is successful,
the products should be delivered to the customer. So the we can perform the whole process with events and events is more understandable for customers also. These could be flow of events like;

• a customer creates an order
• the customer receives a payment request
• if the payment is successful the stock is updated and the order is delivered
• if the payment in not successful, rollback the order and set order status is not completed.

This is more humanly readable and, if a new business requirement appears, it is easier to change the flow.

AWS Serverless Microservices for Ecommerce Application Architecture

Here you can find the main overall Serverless Architecture for our application. This is the big picture of what we are going to develop together for AWS Serverless Event-driven E-commerce Microservices application that is Step by Step Implementation together.

Serverless Event-driven E-commerce Microservices Architecture

Let me try to introduce Serverless components one by one;

Api Gateway

This is entry point of our microservices. API Gateway provides Restful API-Driven Development and Synchronous Event Sources. Synchronous commands are request/response. API Gateway is a synchronous event source and provides a serverless API proxy to Lambda. API Gateway Redirects to CRUD request to internal microservices.

Product Lambda Microservices

It performs CRUD operations using DynamoDB table over the AWS API Gateway. This will cover product table operations fully Serverless in microservices architecture. Synchronous requests will manage by AWS API Gateway and routing requests to Product Lambda Microservices that perform CRUD operations.

We will develop Lambda functions with using AWS SDK for interacting other AWS resources for example in Product case we will interact with AWS Serverless DynamoDB to perform all crud operations.

Basket Lambda Microservices

It performs Add-Remove synchronous basket operations with using AWS API Gateway and DynamoDB. Again Synchronous requests will manage by AWS API Gateway and routing requests to Basket Lambda Microservices that perform CRUD operations. We will write Lambda functions with using AWS SDK for interacting other AWS resources for example in Basket case we interact with AWS Serverless DynamoDB to perform all crud operations.

But also, Basket microservice triggers to event-driven use case which is the checkout basket. When checkout basket, this will publish and create event to Serverless Eventbus which is Amazon EventBridge. So this asynchronous communication will held by Basket Lambda Microservice and Amazon EventBridge and consumed by Ordering microservices over the AWS SQS.

AWS Event Bridge

So asynchronous communication held by AWS Serverless Eventbus service which is Amazon EventBridge. We will create Rules and Target definitions for Amazon EventBridge from Basket Lambda microservices. That means we will develop Basket Lambda Microservices when publishing checkout message to Amazon EventBridge with using AWS SDK for development purpose.

AWS SQS and Ordering Lambda Microservices

So after publishing checkout event to the EventBridge, this event will consume by Ordering part. EventBridge send to event to AWS SQS in order to gain power of AWS SQS queue.

After that Ordering lambda microservice will consume this event with polling. That means we will use event source mapping communication type here when consuming events, ordering lambda microservices send polling request and get event from the AWS queue.

After consuming the event from the AWS Queue, Ordering lambda microservices process the event with creating order record into its DynamoDB table. Ordering lambda microservices perform all these operations with developing lambda functions with using AWS SDK.

Microservices Communications with AWS Lambda Invocation Types

We have 3 communication types;

Serverless Event-driven E-commerce Microservices Invocation Types

1. Synchronous Communication with AWS API Gateway for routing request from client applications to downstream microservices.
2. Asynchronous Communication with AWS Serverless Eventbus which is EventBridge for applying Event Driven asynchronous Communication patterns.
3. And lastly we have Event Source Mapping Communication when polling queue records from lambda services to AWS SQS-Simple Queue Service for Decouple Microservice and processing events asynchronously.

When designing our application, we will follow these Microservice communication types with Lambda invocations, and develop our sections following these arrows that you can find on the above diagram.

Create Infrastructure for Event-Driven Microservice Architecture with EventBridge, SQS and Lambda

we are going to Create Infrastructure for Event-Driven Microservice Architecture with EventBridge, SQS and Lambda. Here you can see the overall architecture that we are going to build in this hands-on section.

Event-Driven Microservice Architecture with AWS

So We will create API Gateway, Basket microservice lambda function, EventBridge custom event bus, SQS — order queue, Order microservice lambda function and notification microservice lambda function. Please follow the default configurations and create one by one:

• Create Lambda function — Order, Basket, Notification. Give permission according to interactions.
• Add Required Permission — Attach Policy — AmazonEventBridgeFullAccess
• Basket microservices
  EventBridge:PutEvents
• Order microservices
  SQS:Poll
  DynamoDB:Put
• Create API Gateway — BasketAPI — POST
• Create DynamoDB tables — Order table
• Create SQS queues — order-queue
• Create Event Source Mapping Polling Invocation — SQS and Lambdas
• Create Amazon EventBridge Customer Event Bus — BasketCheckoutEventBus

Amazon EventBridge Customer Event Bus

Before we start, we should analysis and design our custom event bus. Basically EventBridge has 3 main core concepts that we need to create;

1. Custom event bus — CheckoutBasketEventBus
2. Event Rule with event pattern — we will create CheckoutBasketRule
3. Targets — we will create SQS order queue and notification ms target for our custom event bus

With this plan, Basket microservice invoke from API gateway with post request, Basket microservice publish checkout basket event to the custom event bus CheckoutBasketEventBus process Event Rule with Event Patterns and publish event to the Target systems.

• Create Rule for Custom Event Bus
• Event pattern:

{
 “detail-type”: [“CheckoutBasket”],
 “source”: [“com.swn.basket.checkoutbasket”]
}

• Targets — SQS Queue — order-queue

Develop Basket Microservices Lambda Function Code

We are going to Develop Basket Microservices Lambda Function Code.

create package.json
npm init -y

Install required packages:

npm install @aws-sdk/client-eventbridge

Now its time to develop our Lambda function code. We should plan our developments for Basket Microservices. Lets write pseo code first:

// 1- redirect incoming http request to correct path
// 2- get request body payload which includes event data
// 3- publish message to Amazon EventBridge Custom Eventbus with using eventbridge sdk package
// 4- return back snyc basket payload to the api gateway

Now we can start code implementation. First thing to create client nodejs module to interact with EventBridge APIs.

• Create eventBridgeClient.js

import { EventBridgeClient } from “@aws-sdk/client-eventbridge”;
// Set the AWS Region.
const REGION = “us-east-2”;
// Create an Amazon EventBridge service client object.
export const ebClient = new EventBridgeClient({ region: REGION });

• And we can plan incoming payload and EventBridge custom Event bus event pattern rules:

Expected payload 1 :
{
 “item”:”iphone”,
 “price”:150,
 “detailtype”:”CheckoutBasket”,
 “source”: “com.swn.basket.checkoutbasket”
}
Event Pattern Rules:
{
 “detail-type”: [“CheckoutBasket”],
 “source”: [“com.swn.basket.checkoutbasket”]
}

Its time to develop actual lambda function:

import { PutEventsCommand } from “@aws-sdk/client-eventbridge”;
import { ebClient } from “./eventBridgeClient.js”;
export const handler = async (event) => {
 console.log(“event:”, JSON.stringify(event, undefined, 2));
// pseo code
 // 1- redirect incoming http request to correct path
 // 2- get request body payload which includes event data
 // 3- publish message to Amazon EventBridge Custom Eventbus with using eventbridge sdk package 
 // 4- return back snyc basket payload to the api gateway 
 
 try {
 
 // 1- redirect incoming http request to correct path 
 if (event.httpMethod != ‘POST’) {
 throw new Error(`Http Method should be POST. You are requesting : “${event.httpMethod}”`);
 }
// 2- get request body payload which includes event data
 // expected request payload : { “item”:”iphone”, “price”:150, “detailtype”:”CheckoutBasket”, “source”: “com.swn.basket.checkoutbasket”
 const basketRequest = JSON.parse(event.body);
 if (basketRequest == null || basketRequest.detailtype == null) {
 throw new Error(`basket detail-type should exist in basketRequest: “${basketRequest}”`);
 }
// 3- publish message to Amazon EventBridge Custom Eventbus with using eventbridge sdk package
 const params = {
 Entries: [
 {
 Source: basketRequest.source,
 Detail: JSON.stringify(basketRequest),
 DetailType: basketRequest.detailtype,
 Resources: [ ],
 EventBusName: ‘CheckoutBasketEventBus’
 },
 ],
 };
 
 const data = await ebClient.send(new PutEventsCommand(params));
 console.log(“Success, event sent; requestID:”, data);
 
 // 4- return back snyc basket payload to the api gateway
 return {
 statusCode: 200,
 body: JSON.stringify({
 message: `Successfully finished basket operation: “${basketRequest}”`,
 body: data
 })
 };
} catch (e) {
 console.error(e);
 return {
 statusCode: 500,
 body: JSON.stringify({
 message: “Failed to perform operation.”,
 errorMsg: e.message,
 errorStack: e.stack,
 })
 };
 } 
 
};

As you can see that we have developed Publish Message to customer Event Bus using AWS SDK, and finished development of Basket Microservices. I skipping other commands but if you would like to continue, you can follow the below course.

Step by Step Design AWS Architectures w/ Course

I have just published a new course — AWS Lambda & Serverless — Developer Guide with Hands-on Labs.

In this course, we will learn almost all the AWS Serverless Services with all aspects. We are going to build serverless applications with using AWS Lambda, Amazon API Gateway, Amazon DynamoDB, Amazon Cognito, Amazon S3, Amazon SNS, Amazon SQS, Amazon EventBridge, AWS Step Functions, DynamoDB and Kinesis Streams. This course will be 100% hands-on, and you will be developing a real-world application with hands-on labs together and step by step.

Source Code

Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.