Event-based Microservices: Message Bus
Simple, Scalable, and Robust
A message bus (or event bus) solves the problem of how microservices communicate with each other. This sounds like a problem that could be solved using simple direct connections, but there are challenges that would come along with that solution. Some examples of these challenges include handling horizontal scaling, network disconnects, and service discovery.
To learn more about where a message bus fits into a microservice architecture, you can view the Event-based Microservices: Overview post.
In short, the event bus is the hub that holds the system together.
Event Structure
First of all, we need a standard way to define microservice events. Events are blobs of data that can be structured in any serialisable data format. The event needs to contain some information on what type of event it is, and the data required to be processed. Below is an example event using JSON.
https://gist.github.com/GVaughanUT/03b9d75fa19e442ec059de12be84858e#file-example_event-json
Message Bus Connection
The event bus uses a publish-subscribe pattern. It allows consumers to subscribe to messages of certain types and allows producers to create messages of a certain type. The bus then connects these producers and consumers together.
The following diagram shows an example of a message bus connecting two services. With red events being consumed by the left microservice and green events being produced. These green events are in turn consumed by the right microservice which then produces yellow events.
One thing you might have noticed is where the initial red events come from, how do they get into the system? Microservices don’t need to always consume input events from the message bus. In fact, input events can be consumed from external sources (database or web app, etc.). And in a similar way, not all microservices need to produce an output to the message bus.
Event Routing
A key feature of the message bus is that it handles message routing. This could be as simple as the top example where the green output is produced by one microservice and only consumed by one other microservice. But we can have as many microservices as we want consuming the same green message. A microservice can also consume or produce as many different types of events as it wants. Below is an example of more complex event routing.
This concept of event routing is where the message bus shows how powerful it can really be. As it puts the responsibility of subscribing to data on the service requesting it, meaning that the overall system can remain decoupled and modular. New sections of the system can be added, moved around, and removed relatively easily; resulting in a flexible solution.
Kafka
The event bus needs to handle more scenarios like service discovery, network disconnects, and multiple instances of the same microservice—quickly becoming a complex problem. Kafka is a technology that solves these problems, allowing a reliable message bus implementation.
Kafka is a distributed queuing system, which allows publishing and subscribing to queues. In a microservice architecture, microservices would subscribe to queues for their input events, and publish to queues with their output events.
You can read more about Kafka from its documentation here.
TL;DR
A message bus facilitates events being routed between microservices; without having tight coupling between microservices.
If you found this interesting you can read more on event-based microservices here!
