Event-driven architecture with microservices using event sourcing and CQRS
Although the title may seem long and complex, I will try to briefly go over each concept :)
As it can be understood from the title, in this article I would like to talk about how to create event-driven architecture with microservices using event sourcing and CQRS.
Today’s popular architecture microservices, as you know, are very successful in solving the disadvantages of monolithic architecture. However, it is possible to encounter some limitations during the integration of the microservice architecture into some solutions.
Now, we will go through a real-life example and make all these concepts more concrete.
Let’s consider a warehouse. For this warehouse system, there will be products and building materials for those products. For instance, dining tables and chairs are products, and construction materials are articles such as screws and wood. If we consider a high-level system design, we can mainly talk about two services. These can be product and inventory services. The product service will take care of creating, deleting, updating, and retrieving product data, and the inventory service will work with articles.
A product can be made of more than one article. Therefore, we can say that between product and inventory data, one-to-many relationship can be established. Now, let’s assume that a product has been sold and needs to be deleted. After the product is deleted, the number of building materials used in the construction of that product will also need to be updated.
Despite running in different containers, the product service and inventory service can access the same database’s tables. As a result, correct transactions with ACID attributes will be used. To ensure correct atomicity, inventory data can be updated wherever product data is.
In a microservices-based architecture, a shared database is not advised because other services could be affected by changes to one data model.
Each microservice needs to have its own database as part of best practices for microservices. We can get around this by combining our microservices components with an event-driven design.
Now, let’s take a closer look at event-driven architecture.
Event-Driven Architecture
First, we need to define what an event is. The event is an important business moment like an item being placed in a shopping cart, transaction or site visit. Events can either carry the state like our example (the product sold, its price, and a delivery address) or events can be identifiers (a notification that an order was shipped).
An event-driven architecture uses events to trigger and communicate between decoupled services.
Event producers, event routers, and event consumers are the three main elements of event-driven architecture. An event is published by a producer and sent to the router, which filters and sends it to consumers. Due to the decoupling between producer and consumer services, each can be scaled, modified, and deployed separately.
Now, if we integrate this architecture into our current solution, any change in the product data will be published as an event to the messaging system, so that the event consumer consumes the data and updates the inventory data for the given product changed event.
This method’s drawback is that it is difficult to handle atomic updates between the database and publish events to the message queue. Although distributed transaction management is capable of handling these transactions, it is not advised in a microservices model because XA transactions may not always be supported.
Event-sourcing can be added to this microservices architecture to get around these limitations.
Event Sourcing
The majority of applications save data into databases together with the entity’s current state. For instance, whenever a user orders a product, the inventory table is updated to reflect availability of articles. We are always aware of the data’s most recent status in this way.
Applying the event sourcing pattern has changed the way data is saved into databases. Event sourcing pattern enables to save all events into database with sequential order of data events rather than saving latest status of data into database. A database for events is called event store. Each modification is appended to a list of events in order, rather than changing the status of a data entry.
With event sourcing we have some limitations. For instance, complex handlings are required for queries on the most recent data or a specific piece of data in the event store. In addition, the model for inserting and requesting data is the same, which could make it more complicated to map to the event storage. Also, to store the entire history of records, the event store’s storage capacity must be increased.
To overcome all these issues, we will integrate CQRS (Command Query Responsibility Segregation) with event sourcing.
CQRS (Command Query Responsibility Segregation)
First, let’s define what command and query are. The command is an action that modifies the application’s state. Additionally, the query is an action that reads the application’s state.
Models are used in an application to represent data. Traditionally, a single model is used for a certain domain and it manages all operations (create, read, update, and delete). The model, however, may become complex and unmanageable as requirements change.
The CQRS design pattern enables developers to utilise several models to read and write data by separating the commands and queries.
We can have simpler models that are concentrated on various use cases thanks to CQRS’ provision of separation of concerns. This can prevent a single model from becoming unmanageable in a complex application due to intricate validation logic for writes and several data transfer objects (DTO) for reads.
Scalability is another factor contributing to CQRS’ popularity. When we have two services, we may separately scale them to deliver excellent performance. This is especially important when readings and writes differ significantly.
Now let’s put everything together and create our final architecture.
