Micorservices: Intra-service communication

Microservice: What is and what is not

Microservices are now days a buzzword. People are moving towards implementing microservices architecture as it has lots of advantages.

A microservice is not just breaking a big monolithic application to sub-applications, it’s a lot more than that. The concept and epicenter of Microservices revolves around creating a self-contained piece of functionality which offers clear interfaces and could have its own internal components.

When we break an application into multiple microservices based on functionalities, the effective communication between microservices becomes a necessity and not just nice-to-have feature.

Intra-Service Communication:

In general, we have two kinds of communication:Synchronous & Asynchronous

Synchronous(HTTP requests) : Synchronous communication means that two or more services exchange information in real-time.

REST APIs are the best fit for synchronous communication.With technologies like Spring REST (Java), Flask (Python), etc; REST APIs are faster to build and implement.

One major drawback with synchronous communication is that it requires all interacting microservices to be up and running round the clock. And, this creates a hard dependency and breaks the purpose of creating microservices.

Asynchronous: Asynchronous communication refers to the exchange of data/information/messages between two or more services without the requirement for all the services to respond immediately. In simple words the interacting services are not required to be up and running during communication. This can be achieved via either Messaging queues or Database polling.

Database Polling: This approach is very simple to implement. The idea is to create a table in database and maintain the request details in a row with request state as a column. One service will add the row to the table with request state with statues like INITIAL or START, and the other service will periodically check the same table for new data or for a request state like INITIAL or START. The services are not communicating to each other directly instead via a common database table/model.

As per my view, it’s the tactical or short term solution for implementing an Asynchronous communication. Coz. the downstream services have to continuously poll the database (a kind of spin lock). And, the source/sender service have to check the Error or Exception statuses for their Retry Mechanism.

Distributed Messaging System (especially clustered): A distributed messaging system is a promising solution for asynchronous inter-service communication. It overcomes the shortcomings of Database polling and provides more cohesion.

Apache Kafka is a well known and robust solution for Asynchronous inter-service communication. There are other solutions as well like Solace, Active MQ, etc. In this post I am focusing on Kafka only.

“Kafka® is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies. “

Kafka provided Topics where producers can publish/send the messages and consumers can listen to messages.

We can consider a topic to be the same as a Database table.

The main benefit is that when we are using a message queue solution we don’t need to keep polling database table to check if we have new data, instead we will always be listening to some particular topics in order to trigger an action. And, Kafka provides APIs for effectively producing and consuming messages.

Kafka topics are clustered, so there is no possibility of losing the messages.

Let’s take a very simple real world scenario: A service responsible for creating new accounts in a banking system which needs to perform background checks and send confirmation email to the user, after the creation.

This scenario can be converted to 3 microservices:

• Microservice (M1) responsible for creating new accounts in a banking system
• Microservice (M2) which is responsible for doing the background verification
• Microservice ( M3) for sending a confirmation email to the user, after the account creation.

Here intra-service communication can be implemented via Kafka.

Once Microservice M1 receives a new request for account creation, it will publish the Message with required consumer details on Kafka Topic.

Microservice M2 receives the message via kafka consumer, and it will perform the background verification and send the result to the Kafka topic.

Microservice M3 receives the message from Kafka topic, it will send the email to the consumer based on the message details- if the account creation is successful, failed or more information is required from consumer .

Benefits: Now, M1 , M2, M3 do not need a synchronous communication and we can achieve below benefits:

1. We can introduce a retry mechanism in background check and email microservices, so that if any of the services fail, the consumer is not required to submit a new account creation request.
2. The services M2 and M3 can be users for other business functionalities as well. Like email microservice can be used for sending reports and monthly statements to consumers.

Thanks for the read. I hope you liked the article!! As always, please reach out for any questions / comments / feedback.