Event-based Microservices: Message Bus

Simple, Scalable, and Robust

George Vaughan
Oct 9, 2020 · 3 min read
Image for post
Image for post
Photo by Mathyas Kurmann on Unsplash

A message bus (or event bus) solves the problem of how microservices communicate with each other. This sounds like a problem that could be solved using simple direct connections, but there are challenges that would come along with that solution. Some examples of these challenges include handling horizontal scaling, network disconnects, and service discovery.

To learn more about where a message bus fits into a microservice architecture, you can view the Event-based Microservices: Overview post.

In short, the event bus is the hub that holds the system together.

Event Structure

First of all, we need a standard way to define microservice events. Events are blobs of data that can be structured in any serialisable data format. The event needs to contain some information on what type of event it is, and the data required to be processed. Below is an example event using JSON.

https://gist.github.com/GVaughanUT/03b9d75fa19e442ec059de12be84858e#file-example_event-json

Message Bus Connection

The event bus uses a publish-subscribe pattern. It allows consumers to subscribe to messages of certain types and allows producers to create messages of a certain type. The bus then connects these producers and consumers together.

The following diagram shows an example of a message bus connecting two services. With red events being consumed by the left microservice and green events being produced. These green events are in turn consumed by the right microservice which then produces yellow events.

Image for post
Image for post

One thing you might have noticed is where the initial red events come from, how do they get into the system? Microservices don’t need to always consume input events from the message bus. In fact, input events can be consumed from external sources (database or web app, etc.). And in a similar way, not all microservices need to produce an output to the message bus.

Image for post
Image for post

Event Routing

A key feature of the message bus is that it handles message routing. This could be as simple as the top example where the green output is produced by one microservice and only consumed by one other microservice. But we can have as many microservices as we want consuming the same green message. A microservice can also consume or produce as many different types of events as it wants. Below is an example of more complex event routing.

Image for post
Image for post

This concept of event routing is where the message bus shows how powerful it can really be. As it puts the responsibility of subscribing to data on the service requesting it, meaning that the overall system can remain decoupled and modular. New sections of the system can be added, moved around, and removed relatively easily; resulting in a flexible solution.

Kafka

The event bus needs to handle more scenarios like service discovery, network disconnects, and multiple instances of the same microservice—quickly becoming a complex problem. Kafka is a technology that solves these problems, allowing a reliable message bus implementation.

Kafka is a distributed queuing system, which allows publishing and subscribing to queues. In a microservice architecture, microservices would subscribe to queues for their input events, and publish to queues with their output events.

You can read more about Kafka from its documentation here.

TL;DR

A message bus facilitates events being routed between microservices; without having tight coupling between microservices.

If you found this interesting you can read more on event-based microservices here!

UserTesting Engineering

Thoughts, stories, and learnings from the Engineering Team at UserTesting

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store