Real time Notifications using Kafka

Introduction

Logged in users would like to see their notifications so that they can take appropriate actions on the tasks. Sometimes the pre-loaded notifications are not just sufficient when user has loaded the page in the browser but they also would like to see the realtime notifications coming to their screens without loading the page itself.

Challenges

The traditional Request response works great where user comes on the page and on the startup the message intended for the user are queried and displayed however the design needs to cater the other needs like high throughput, realtime notifications even when the page is not refreshed and HA.

The main challenge to display the notifications are for some way the server should understand its client (Browsers in this case) and send the right message to intended user (Browser where she logged in)

The socket connection between the browser and the underlying service can make it possible for this two way communication where user logs in the website and browser gets connected to the backend service. The socket connection are made unique to each browser and a client id (Browser id) is generated by which the backend service understands which user (from which browser/client) has connected. This information is stored in a highly available in-memory cache, like Redis as shown in the picture, for any of the other nodes to read. Which means a “user” coming from a specific browser “client” is connected to which “pod” is stored.

At this moment, the unread messages are queried from the system and displayed to the user. However user is still browsing/or idle and some of the important events are triggered which should be displayed to the user. So this event, whether by some batch processing or realtime generated events needs to be displayed. This is achieved by Kafka which is highly scalable, reliable messaging system. The messages are generated and message headers/data contains all the information e.g the indented user to whom this message should be displayed, the original system which is generating this message, the module and finally the content of the message in payload.

Once this message is retrieved they are pushed to Kafka which Notification Service can consume. Notification service would inspect the message details and understand the intended user for which this is for. It connects to the Redis system and checks the heartbeats if the user is still connected. If it is, then it would find the client id (Browser) and the system Pod or some other (would be explained in the second part) would push these realtime messages to specific browser/client/user.

The messages are also stored in a persistent storage for future use.

The design takes care of the high availability by clustering of service, kafka brokers and Redis system to ensure the messages are delivered in fast and reliable manner.