Microservices Communication Architecture Patterns

1. Problem :

• Multiple Concurrent requests to a service
• Difficult to predict the volume of requests
• Delay in Response is acceptable
• We don’t want consumer service to suffer any performance or reliability issues as it might be used in other places

Solution : Queue-Based Load Levelling pattern:

• Introduce an Asynchronous Queue
• Allow Service to handle the messages at its own pace

Other Benefits:

• Controls Costs: don’t have to design your service to meet peak load

Follow Up :

What if some of the requests/tasks are high priority ? Multiple Queues or Priority Queue

What if we need a synchronous response ? I don’t know answers, but some solutions might be

• Scaling service B
• Rate limit Service A

2. Problem :

• Service(A) is receiving multiple requests simultaneously It is using resources to server each request.
• One of its downstream services(B) is slow/not responding. So The resources used by A’s request may not be freed in a timely manner.
• As requests to the service B continue, those resources may be exhausted. For example, the A’s connection pool may be exhausted.
• At this point, requests by A to other services are also affected.
• Eventually A can no longer send requests to other services, not just the original unresponsive service B.

Solution : Bulkhead Pattern

• Idea : Elements of an application are isolated into pools so that if one fails, the others will continue to function.
• Named after the sectioned partitions (bulkheads) of a ship’s hull. If the hull of a ship is compromised, only the damaged section fills with water, which prevents the ship from sinking.
• Projects like Polly offer a framework for creating bulkheads.

Other Benefits: Isolates critical resources from standard consumers.

Drawback : Less efficient use of resources.

3. Problem :

• Service Calls to service C is failing due to transient faults or temporarily unavailable.
• Pointless for an application (B) to continually retry an operation that is unlikely to succeed
• Retrying might be like bombarding C
• Timeouts might hold lots of concurrent requests

Solution: Circuit breaker pattern

• A circuit breaker pattern is implemented on the caller side(B), to prevent overwhelming a service which may be struggling to handle calls.
• A circuit breaker acts as a proxy for operations that might fail.
• The proxy should monitor the number of recent failures that have occurred, and use this information to decide whether to allow the operation to proceed, or simply return an exception immediately.
• Tools like resilience4j, hystrix can help