Microservices integration in AWS
Orchestrated microservices vs. Choreographed microservices
Choosing between monolithic and microservices is always challenging when starting a new project. But if you decide to go with microservices, the challenges aren’t over. One primary concern you must consider is how the microservices will communicate with each other. Answering this question will guide you on how to develop microservices and which services and tools to use. In this article, we will have a study case and check some alternatives to implement our microservices integration strategy using AWS services.
Our study case
Let’s start defining a study case to keep things more tangible. This time, our study case will be an airline’s booking seat system. Let’s imagine that to reserve seats, we would need at least four microservices: one to manage the available/reserved seats, one to handle the payment, one to create and send the receipt, and one to keep the booking status.
Now, let’s think we received a booking order. The first step is to book the seats. After that, the payment needs to be processed. Finally, if everything goes fine, the receipt should be sent. We also should be able to get the order status at any time. The happy flow is always beautiful, right?
Still, we also need to consider at least one alternative flow. Let’s imagine that the payment is denied. In this case, the system should cancel the booked seat, and the status also should be changed. After deciding on our study case, it is time to discuss how we can integrate those microservices.
There are several strategies to integrate microservices. I chose to talk about two types that I consider more valuable: orchestrated microservices and choreographed microservices. In the following sections, I will show the advantages and drawbacks of each one. I will also show how we can implement them using AWS services.
Orchestrated microservices
In the orchestrated microservices pattern, you will have a predetermined workflow manager. He will be the one responsible for triggering the microservices in the correct order. Let’s have a look at the image below:
As you can see in the image, two flows are orchestrated: the happy flow and the alternative flow. In the happy flow, we consider that everything goes fine when booking the seats. In the alternative flow, we assume that the payment is rejected.
In the happy flow, our orchestrator would first book the seats in the Seats Management microservice and then change the order status. Afterward, it would trigger the payment in the Payment Management microservice and change the status again. Finally, it would ask for the receipt in the Receipt Management and change the status one last time.
The alternative flow is very similar, except for the steps after triggering the payment. After the payment is rejected, the orchestrator must cancel the booked seats in the Seats and change the status again.
Some advantages of this approach
- It is easy to trace the process because it is described in the orchestrator.
- It is easy to change the business flow if necessary.
- The microservices are decoupled and isolated. You can change a microservice behavior or add another service without affecting others.
Some drawbacks of this approach
- Your orchestrator can be a single point of failure.
- The orchestrator can be complex and challenging to maintain as you can have many errors in the flow.
- If the orchestrator breaks in the middle of a flow, you can have a system with inconsistencies.
- The orchestrator has a strong coupling with the microservices. It needs to know about their deployment and interfaces.
- It is hard to work with several teams together, as you have an orchestrator as a single point between several teams.
Implementing on AWS
Since I’m a Serverless enthusiast, I would start by implementing the microservices using AWS Lambda functions:
AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. (https://aws.amazon.com/lambda/)
But of course, if you have a scenario where you can’t use Lambda Functions, you can go with containers in ECS, EKS, or any other solution.
The orchestrator could be another microservice written in your favorite language or framework. But, if you want to save some time, coding, and money, you should consider the AWS Step Functions:
AWS Step Functions is a low-code, visual workflow service that developers use to build distributed applications, automate IT and business processes, and build data and machine learning pipelines using AWS services. Workflows manage failures, retries, parallelization, service integrations, and observability so developers can focus on higher-value business logic. (https://aws.amazon.com/step-functions/)
With step functions, you can quickly build, visualize and trace all the workflow steps. It is scalable and reliable. Also, you can integrate your microservices directly with other AWS services (lambda functions, DynamoDB tables, and many others) and with some external services.
You can check below how our orchestrator could be designed with step functions:
I need to mention that after you draw in the AWS console and test it, you can export the CloudFormation code. I encourage you to take a look at the Step Functions FAQ.
Choreographed microservices
In this pattern, each microservice knows what to do when a specific event happens. The microservices will integrate through a message broker. It will publish an event when necessary and subscribe to others so they can execute their jobs.
In our example, everything would start with the “Booking Request” event. All microservices would process this event:
- Seat management microservice: It should start the booking process;
- Payment service microservice: It should start processing the event;
- Status management microservice: It should create the status record;
- Receipt management microservice: It should get the necessary data and wait for the other events.
After the seat management microservice finishes its processing, it should publish the event “Seat Booked”. This event would be processed by:
- Status management microservice: It should update the status;
- Receipt management microservice: It should get the necessary data and wait for the other events.
After the payment management microservice finishes processing, it should publish either a Payment OK event or a Payment Failure event. The “Payment OK” should be processed by:
- Status management microservice: It should update the status;
- Receipt management microservice: It should start processing the receipt with the information provided by all received events.
The “Payment Failure” event should be processed by:
- Status management microservice: It should update the status;
- Seat management microservice: it should free the booked seat.
Some advantages of this approach
- You don’t have the orchestrator as the single point of failure.
- You can easily restart the process by continuing the queue.
- Since you don’t depend on a central orchestrator, you can have different teams working on the microservices.
Some drawbacks of this approach
- You can’t quickly check the implemented business flow logic.
- It is harder to troubleshoot the issues or trace the status between the microservices.
- You can have some logic from the business flow on the microservice, or you need to add some intelligence to your event bus.
- You may need to wait for several events to continue your process in a microservice.
Implementing on AWS
There are different tools you can use as a Message Broker. One of the most known is Apache Kafka. You can build your cluster or use the AWS MSK service. But for this article, I would like to mention two other services to implement a Publish/Subscribe pattern.
- Amazon Simple Notification Service (SNS): Your microservices should publish and subscribe to specific topics using this service. It supports 2,5 million subscriptions for standard topics and 100 for FIFO topics. It doesn’t support Schema Registry or Event Filtering. The delivery guarantee is at least once for standard topics and exactly once for FIFO topics.
- Amazon EventBridge: Using this service, your microservices would publish an event to an Event Bus, and to receive them, you should create rules based on the content. If the content matches the rule, it will be delivered to the subscribers. It supports 300 rules per event bus and five targets per rule. It supports Schema Registry and Event Filtering. It doesn’t support FIFO order, and the delivery guarantee is at least once.
Both services integrate with AWS services, and the SNS can also directly send e-mails or SMS messages. Don’t forget to check the SNS FAQ and the Event Bridge FAQ.
Which strategy should I use?
Unfortunately, there isn’t a silver bullet choice. For the starting phase of a project, I would start with orchestrated microservices using AWS Step Functions. And later, if more teams need to work on the same project, I would change the strategy to choreographed microservices. By doing that, we would increase the complexity only when necessity should arise.
The other alternative is to start your microservices using the choreography strategy, communicating through SNS or EventBridge from scratch, for example. Still, it might add unnecessary complexity to your system from the start if you don’t have more teams working on it.
As I said, considering both paths, my option would be to start with orchestration. Still, you must remember that you might have to change your approach if your service grows.
Do you think you have what it takes to be one of us?
At WAES, we are always looking for the best developers and data engineers to help Dutch companies succeed. If you are interested in becoming a part of our team and moving to The Netherlands, look at our open positions here.
WAES publication
Our content creators constantly create new articles about software development, lifestyle, and WAES. So make sure to follow us on Medium to learn more.
Also, make sure to follow us on our social media:
LinkedIn — Instagram — Twitter — YouTube
