What problem will be solved/created by changing architecture to CQRS and Event-sourcing system
If you are planning to build a high-concurrency system, or when the simple server can not properly handle all requests while the user and business grow, maybe only refactoring code, tuning SQL script, adding DB indexing, and bringing in Redis cache still do not help.
Then we might consider adopting some powerful architectural patterns to increase the RPS (request per second) that the server can handle. But these patterns are always a trade-off between performance and complexity, so this article will go through the pros and cons of CQRS (Command Query Responsibility Segregation) and Event-sourcing system architecture.
Original System (Simple system)
First, Let's see the simplest system architecture, we receive the command and query request, and the server might call some external service through API then insert/update to DB or just query data from DB.
Pros:
- Simple
easy and quick to implement and maintain
Cons:
- Coupling and Synchronous
RPS and stability might be affected by external API (which we can not control). - Centralize DB
DB becomes a bottleneck because transactions and DB lock cover most processes and affect other requests. - Inflexibility server
Services logic is bound in one server, loss of some flexibility on deployment/scaling according to system characteristics (insert/update heavy or query heavy).
CQRS System
Second, we come to the CQRS system, we divide the DB into the write and View DB to handle insert/update and pure query respectively. and we also need to consider the synchronization mechanism between two DB (an easy way is that we can use the secondary DB in a cluster as the view DB and the primary DB as the write DB).
Pros:
- Separated server (*Solve old cons)
Read and write servers can be deployed/scaled respectively according to business and resources situation. - Separated DB (*Solve old cons)
Read and write SQL can be processed respectively, decreasing the effect of lock on performance. - Separated schema
Sometimes can use optimized schema for both the write and read in their own DB (be aware, this will increase the complexity of synchronization).
Cons:
- Coupling and Synchronous (*Still exist)
- Eventually consistency (*New problem)
Data is eventually consistent. query side needs to wait until the synchronization
CQRS + Event-Sourcing System
Finally, we adopt the event-sourcing pattern base on the CQRS system (not necessary, but mostly combine these two patterns together, it can bring out the most benefit). The architecture changes a lot, so let us go through the responsibilities of each component.
Command Server:
- Accept command requests and send an event.
- If need the current state of data can query from the query server.
Event Handler Server:
- Listen to events and call some external services then send a new event or send a new request to the command server.
Query Server:
- Listen to events and update its own view.
(maybe a DB, in-memory cache or some data persist component) - Accept and respond to query requests.
Event Store
- Store all events.
Event Bus
- provide events to its subscribers
(consumer pull events or Event-Bus push events are both acceptable, depending on what component we use. i.e. Kafka, RabbitMQ…etc).
(In my implementation, I use Kafka to be both the Event Store and Event Bus, the POC project can see at https://github.com/NoahHsu/event-sourcing-order-poc)
Pros:
Besides all pros of CQRS,
- Asynchronous and Decouple (*Solve old cons)
Build an asynchronous, decoupled way to communicate with external services, which increases the RPS of our system and makes the system more reliable. - Plugable feature
A new feature can be plugin by just adding the listener of some event, no need to change the API interface or integrate API so often. - Single source of truth (*Refine old pros)
Obtain data states which are projected from one single source of truth, the event store. make the separated schema easy to sync. - Audit log
Provide a complete audit log for all events that occurred in the system without extra effort and - Easy data migration
No need for a data migration script, just deploy the new logic and replay events. - Better performance (*Solve old cons)
Events are only appended to the end, so there is no update will occur on the system data.
Cons:
- Eventually Consistency (*Still existing)
Data is eventually consistent. query side needs to wait until the event is consumed. - Design Change (*New problem)
Changing design thinking and method. e.g. Event storming, event-driven, event-replay, - More Complexity (*New problem)
Besides the maintenance effort for event-handler, event-store, and event-bus, the event-replay mechanism needs extra design when the event number grows (e.g. snapshot for the projection and dividing a history event store).
Conclusion
After a briefly analyze, we can see that the three system architecture designs have their own strength and weakness. The more powerful ability on handling high concurrency requests the more complex it is. So there is no silver bullet, we need to find a way that mostly fits our business scale and resources limit.
further research
If you found this article valuable, please consider supporting my work by buying me a beer 🍺. Your generosity fuels my passion for creating more Event Sourcing and CQRS system content. Don’t forget to share and clap for this article, and follow me on Medium for future tech stories. Thanks for your support and happy reading!
