Aggregation Pattern (Design Patterns for Microservices)
If you’re new to microservices, I recommend reading my prior article to learn about some basic microservices principles.
- Introduction to Docker and Container-based development. 🐋
- Web application to Microservices (Do we use it or not?). 💻
- Best Practices for Microservices.✅
So, if you’re good with microservices, get ready to learn about microservice design patterns. In this article, I'm going to explain to you about Aggregation pattern.
The aggregator design pattern describes a service that gets a request, then makes several requests to different services, combines the results, and then responds to the initial request.
Aggregation Pattern has 03 different branches (03 ways that you can implement aggregator pattern)
😮 Use case:
Mercantile Finance is a global financial services firm. Monolithic architecture is used in their current employee management system. The IT team at Mercantile Finance decide to migrate their system to microservice architecture because microservices are trending and eliminate most of the difficulties with monolithic applications. Mercantile Finance decided to outsource their system because it is vast and complex for their IT team.
You work as a software engineer for the IT firm where Mercantile Finance’s system was that outsourced. So, for Mercantile Finance’s employee management system, your Tech lead wants you to implement some microservices. So, using DDD (Domain Driven Design), you came up with the services listed below for the system.
🔩 Service 1: get personal information.
🔩 Service 2: get leave information.
🔩 Service 3: get employee performance information.
🔩 Service 4: get allocation information (which project employee is working, what are the work allocated).
💠 So now you have 04 different services to implement. There are some other existing services as consumers implemented by the Mercantile Finance IT team. These existing consumer services are attendance management system and project management system.
💠 As the next step, you decide to go ahead and implement these micro-level services, then create dockers for this service and deploy them.
💠 Now here comes the problem…! 🤔
- The attendance system doesn’t have a service to get personal information and leave information.
- The project management system doesn’t have a service to get both personal information and allocation information.
💠 This is because these services alone won’t do anything except what these services designed to do (“Do one thing, do it damn well”)
💠 As a solution, you can create an aggregate service (You could also create a proxy) that consumes 02 microservices and give a response back to a consumer.
Ex: Create a new service to consume fetch personal information service and fetch leave information service then send the response to the attendance management system.
💠 So, this newly created service will receive the request from a consumer, then it will invoke personal information service and fetch leave information service.
💠 Then the newly created service will aggregate responses from these two services and will send the response to the consumer.
💠 This is how aggregator basically works. Now let’s see how aggregator can be implemented with parallel and scatter and branch pattern with related use cases.
Parallel aggregation (Scatter gather pattern)
💠 In Parallel aggregation, requests are sent to necessary services parallelly. Then the responses are aggregated and the response is sent back to the consumer.
😮 The use case for Parallel aggregation
💠 When attendance service invoked parallel calls are sent to personal information and leave information. Get responses, aggregate them and sent them to the attendance service.
💠 When the attendance service is invoked, you can send parallel calls to personal info. service and leave info. service. Then get responses from both services and aggregate them as a single response to the attendance mgt. system(consumer).
Chain pattern (Service chaining)
💠 Unlike parallel aggregation in the chain, pattern requests are not sent to services parallelly.
💠 Assume there are 3 services named A, B and C implement with chain pattern (As in diagram below). How it works according to chain pattern is, consumer first sends the request to service A, then service A sends a request to B to and finally to service B sends a request to service C (consumer > A > B > C). The best practice is to keep the chain length to a minimum.
😮 Use case Chain pattern
💠 Using chain pattern when a service has a dependency on another service.
💠 The leave system is still in a previous version where it's not aware of fetching data with employee id (empId) in DB. Instead, it sends employee code (empCode) to fetch data.
💠 So, what you can do as a solution is, you can invoke personal info. service and get employee code (empCode) from that. Then pass the employee code (empCode) into leave info service (with along other information) and get leave information. And now you can send the aggregated response back to the consumer.
💠 It's not necessary to get something from the first service, but one after the other.
Time consumption in parallel and chain pattern ⏰
💠 Assume that, personal info. service consumes10ms and leave info. service consumes 10ms.
💠 If we use parallel aggregation, you can get both information (theoretically only, practically not possible) in 10ms and may deliver the response in around 15ms to the consumer.
💠 If we use the service chaining approach (one after other) what happens is 1st call will take 10ms, 2nd call will take 10ms and altogether it will take 20ms and it could take around 25ms to deliver the response
Branch aggregation
💠 Based on a factor, a decision is made about where the service should go next. Branch aggregation can be used to call different chains, or a single chain, based on the requirement.
😮 Use Case Branch aggregation
💠 There is a service where users send user id (userId) and branch code (branchCode). This request goes to the personal info service.
💠 Then, If the branch code (branchCode) is “sales”, the request next goes to sales info. service or if the branch code (branchCode) is “engineer”, the request should go to technology info. service.
💠 The major advantage of using Aggregator Pattern is flexibility for requirements.
🤔 Use case
💠 You implemented above mentioned 04 and other aggregation services for the attendance system and project management system. Now everything is complete and you are in production.
💠 But later project management teams decide to examine employees leave information before allocating an employee for a critical project. The business logic of the system changes as a result of this).
✅ Solution
💠 You can create a news aggregation service. So, this newly created service will invoke all 3 services (Personal info. service, leave info. service, allocation info. service).
💠 Now you have 2 different aggregated services for the project management system. So, you can deploy both the services to production. Then you can let the consumers know there is a newly updated service and you can migrate to that one.
Did it occur to you that you didn’t need to make any changes to the existing main 04 services for this requirement? Cool right… 😎
💠 So, now consumers can (according to their timeline) migrate to the newly created service. After all, consumers migrated (you can monitor logs to see if the old service is still in use) you can shut down the previous service and keep the updated version live.
Because Aggregation services do not perform any processing or any transformation, business logic. They do not require much computing power.
Authorization and authentication flow
💠 When implementing aggregated services, you should be very mindful of how authorization and authentication flow.
💠 Assume you’re validating requests with OAuth2. So, if each service validates your token and each validation takes 5 milliseconds, the newly created aggregated service will take 15 milliseconds to validate your token. As a result, doing token validations on different services is a time-consuming task.
💠 Therefore best practice would be to have a separate identity management service and redirect traffic (from the aggregation service) through that identity management service…
Since every second counts in a response cycle, you should aim to save as much time as possible on your round trip.
💠 Finally, make sure that there are no vulnerabilities when you are introducing new authorization and authentication mechanisms to your services.
Anyway, that’s all for this article. In the future, we’ll talk more about design patterns. Please share your ideas and suggestions in the comments section below. Thank you very much.
