Dispelling the Eventual Consistency FUD when using Event Sourcing

So your organization is toying with the idea of using Event Sourcing, the benefits look promising but you can’t seem to shake off the nagging doubt of that one aspect of adoption — Eventual Consistency because of its alignment with CQRS. Your organization’s systems have always been following the principle of Solid Transactional Consistency (maybe you are in the Banking Business) and CQRS/ES becomes a big no-no because of just this one aspect. Everyone within your organization finds those examples of order items eventually appearing in your cart extremely amusing. Finally, something to agree upon on that Friday Zoom bonding call!

We in the CQRS/ES space are equally guilty of not countering the FUD with enough examples and literature that gives relevant data points for folks wanting to adopt these patterns. What if I told you that your systems built with CQRS/ES are as transactionally consistent as systems that are built without these patterns using the more formal methods of storage!

Well, let’s walk through an example to actually prove it. Before we get into it, a quick 30,000 feet view of these patterns — CQRS helps you split your Domain Model into two distinct partitions, the Command Side for processing “Commands” which result in a state change and the Query Side for processing “Queries” to retrieve state. Event Sourcing proposes the storage of the state of your application as a sequence of immutable events in an event log — State changes are validated directly against that event log, while Queries are executed against projections derived from that log. This allows the event log to be considered a reliable source of truth.

We will take probably the most commonly used example of Money Withdrawals to demonstrate this. The only invariant that we need to take care of is that the customer has the required balance in their bank accounts to perform a withdrawal. A customer might make multiple withdrawals in a short time frame and at the end of each withdrawal the balance needs to be in a consistent state. Else you could have customers make more withdrawals than allowed and the bank shuts shop.

Now, traditionally you deal with the consistency problem by using a database with a single mutable record representing the account balance. Every withdrawal operation loads the balance record, checks the withdrawal amount against the current balance and accordingly the system makes a decision whether to permit the withdrawal or not. An important point to note here is that before the customer makes an intent to make a withdrawal, they are presented with a View of their current balance. This View is constructed by loading the same mutable Account Balance record.

Essentially you use the same model:

• To display the balance which helps the customer to make an intent to do a withdrawal.
• To help the system make a decision on whether the intention to make a withdrawal is permitted or not.

This approach gives you the perceived benefit of guaranteed “read” consistency. While the information is presented on screen, there is no guarantee that the data isn’t modified in the meantime. The user’s decision is always made on stale data. In addition to that you have to deal with problems around scalability, availability and model complexity. CQRS/ES offers a great way to help address these kind of problems with strong consistency guarantees.

How exactly is that achieved ? Going back to our original example, say we now have decided to adopt Event Sourcing. So instead of a single mutable account balance record, we now store all the operations (All the withdrawals and the deposits) that happen against that account as a sequence of events.

When the customer now makes the intent (i.e. a Command) to do a withdrawal transaction against a particular account, this will load the past events from the Event Store against that account (i.e. the Withdrawals/Deposits) and then replay them to arrive at the current state for the account balance. This state is then used to make the decision on whether to permit the withdrawal or not. In short, it is guaranteed that Command processing in an Event Sourced system will always be done against the latest and current state not an eventual state.

Which takes us back to the View as the customer will make the intent to perform a withdrawal only after they are presented with the balance. In Event Sourcing based systems the balance (aka the balance projection) that needs to be displayed is not retrieved from the Event Store. It is retrieved from another datastore which constructs and stores the balance the way it is used in this case as a display. The construction and storage is done in an asynchronous fashion by subscribing to the Events that are emitted once it is committed to the Event Store. Essentially it might not reflect the current state and hence we term the balance projection as eventually consistent.

To summarize, Command Processing in Event Sourcing is always going to be performed against the latest state and not an eventually consistent state. Projections which are used to query the state are always eventually consistent owing to the asynchronous nature of its construction and storage. While the example shown here is a bit trivial to demonstrate the concept, it holds true for very complex command processing too. It should be noted that in any system being built there will be islands of consistency in a river of constant change. This holds true for both Strongly Consistent and Eventually Consistent systems.

Axon provides Event Processors which significantly scale up the construction of Projections in almost near real-time once events have been published. In addition, it also provides Subscription Queries which listen to any updates to your Projections and delivers it in real-time to interested clients resulting in an enhanced user experience.

Website: https://axoniq.io/

Youtube channel: https://www.youtube.com/c/AxonIQ

Reference Guide: https://docs.axoniq.io/reference-guide