Event Sourcing Part IV: Evolving Your System
In Part III of this SSENSE-TECH series, I discussed how to expand your Event Sourcing application by introducing a way to handle different access patterns and other performance-related issues, such as long streams. In this final part, I will present the remaining topics to conclude this broad overview on Event Sourcing.
Evolving Events
“The only constant in life is change.” — Heraclitus
It is a fact that your system, no matter how well designed, will evolve and change as new requirements arise. These changes will force you to rethink and change, or introduce new concepts in your system. If we recall that the state of our application is expressed with events, it is clear that those events will also have to evolve in some way.
There are many ways to handle this and I will discuss the most common one while highlighting the advantages and rules to follow.
Before we go over the details, here’s one recommendation: sometimes, changes in your system can be better encapsulated as new events instead of modifications to an existing one. If you evaluate and apply this carefully, you will save considerable time by avoiding unnecessary changes that will make your existing events more confusing to understand.
As a general rule, you should be able to move from the previous version of the event to the new one. If you can’t do this, you are looking at a new event.
Mapping
Imagine that our e-commerce solution is now offering multiple delivery methods for the customer. After evaluating the current state, you decided that the Order entity will be updated to contain the delivery method selected by the customer.
Previous to the change, when the customer created an order, an OrderCreated event was published with the following content:
Now, due to the change, it will look like this:
When reconstructing the state of the Order entity our apply method would now look like this:
Problem solved, right? Not quite… We are going to have problems if we keep the code as is. What happens when you release this change and you want to retrieve an Order that was created before this update? Your code would fail, or at least behave unexpectedly; it would try to access the non-existent property deliveryMethod on the already existing OrderCreated events.
A non-Event Sourcing solution could be to write a database migration to update the structure and set a default value to the new property. This approach has issues on its own, but for Event Sourcing systems this would violate the immutability of events, which is the cornerstone of this pattern.
To solve this limitation, we will apply versioning to the event and convert it from the previous version to the current one.
This works well considering it can detect the version of the event. By knowing the changes, we can apply, for example, a default value to the previously non-existent value. But we can do a bit better!
This updated version has the benefit of keeping your domain unaware of previous versions, as it only receives the latest representation of the event. All logic associated with the conversion/mapping is moved to a dedicated component. As a recommendation, consider adding a version field into your events from the beginning and increment it as you make changes.
Versioning the Domain Logic
When we look at the evolution of your system over time, its changes can contain more than new properties. The logic used by your domain can evolve as well, and it is important to capture this logic in the events as they change.
In our domain, imagine that as part of our Order we have to express how much the customer will be charged in a specific currency. Imagine we captured the logic by taking the amount in a base currency and applying an exchange rate.
This works fine until you have to replay your event stream. Because the exchange rate that will be used is whatever the market has at the moment of the replay, the results you will achieve are not going to match what the customer was charged.
To solve this issue, one solution is to include the exchange rate that was used in the event and in this way “freeze” that information in time.
As general advice, if you have to make calls to external services to retrieve information to update the state of your entity, add the information retrieved in the event. This will allow you to achieve a deterministic replay of the events without any external dependencies.
GDPR and the Right to Be Forgotten
General Data Protection Regulation (GDPR) is a regulation law created to protect the citizens of the European Union, it addresses how companies should handle personal data. The law introduced a lot of changes to the systems in order to comply with the requirements stipulated. In a simplified way, companies have to be able to provide, upon request, the personal information they contain on their users and also satisfy the right to be forgotten.
This last stipulation is particularly complex as it involves potentially deleting information that affects the integrity of the system it contains. Imagine, for example, that you have a system that sends messages from one user (A) to another user (B). If user (A) requests to be forgotten, what will you do? Delete the message user (B) sent as well? All of a sudden user (B) will complain something disappeared.
As we can see, GDPR has profound consequences and while none of them are exclusive to Event Sourcing applications, it is important to mention that because in the Event Sourcing pattern, state and data are handled in a different manner than in a traditional application, the solutions you adopt to comply with GDPR will be different.
Immutability
One of the basic aspects of Event Sourcing is the fact that you reconstruct the current state by replaying the events. For this reason, it is expected that an event, once written, is immutable. When a user requests to be forgotten, and you have a traditional state persisted, you could address it by anonymizing the records on your persistent medium for the user. How do we solve this when, in theory, you should not or cannot alter the events associated with that user?
Encryption of the Personal Information in the Events
In order to comply with GDPR, you must guarantee that you will not be able to reconstruct the personal information associated with the user. A possible solution involves using encryption of the personal data existing in the event before persisting to the Event Store. The encryption, as illustrated in figure 3, requires a key that is stored in another component and not persisted with the event. This component is commonly referred to as a Key Management System (Service) or KMS.
When the user requests to be forgotten, you will delete the key associated with that user from the Key Management System. By making this change you prevent the replay of the event stream to be capable of reconstructing the full state of the entity, as it can’t decrypt the personal data.
Your application needs to provide some default values to be able to fill the data it could not decrypt so the application logic still behaves as it should. This solution, although simple in concept, adds another component (the KMS) which can incur more complexity and costs on setting and maintaining it on the scale.
Having Two Streams, One with Public and Other Private Information
One alternative solution to our problem is to split your stream in two for the same entity, one containing the private data and another with the public data.
When the user decides to be forgotten your system will delete the public stream associated with them. This way when you want to reconstruct the state your Event Store will only be able to provide the private stream. This solution aims to comply with the GDPR requirements without adding another external component to your system.
Please note that, like the encryption solution, you will have to modify your application to handle the now missing information and fill it with default alternatives that allow it to continue to operate as expected.
The downside of this solution is having to split the produced events in up to two separate streams per entity, which adds more complexity to the persistence and replayability of the events.
Projections
In Part III of this series, I presented projections as a way to efficiently satisfy the need for different access patterns. Projections potentially contain personal information of a user and therefore must be addressed as well.
Projections are used to satisfy read-only scenarios and are created by subscribing and manipulating the events produced by the write side of your application. Given this nature, you can safely delete the data without major repercussions.
One recommendation is to consider using the same streams to propagate the compliance to the right to be forgotten. With this in mind, your application would emit one special event as it executes the right to be forgotten.
Your projections would receive this event and contain the necessary logic to delete any personal data available for the user.
The End… Or Is it?
Event Sourcing is an exciting alternative to a more traditional approach and fits perfectly within most use cases that our distributed, microservices-based, and eventually consistent environment presents. This series is aimed at presenting a 360 degree view of what Event Sourcing is, while also discussing the practical considerations you should be aware of as you decide to adopt this approach. I invite you to continue your journey and explore this subject in more detail, and I list below some resources that I find relevant. Enjoy!
How to Version Event Sourced Systems
A free book by Greg Young with various approaches that can be used to handle the fact that your events will likely be changing over time as your system changes as well.
A free book that presents an in-depth journey on developing a system that uses CQRS and Event Sourcing. Don’t be fooled by the Azure mention as the concepts and examples are still relevant for any cloud provider or underlying technology.
Event Storming Modeling Patterns: Going Beyond Superficial
Introduction to the use of a modeling/analysis technique called Event Storming and some useful patterns when approaching the development of any Event Sourced solution.
Effective Aggregate Design: Part I, Part II, and Part III
Aggregates are the base of any Domain Driven Design (DDD) systems and this 3-part series contains a laser-focused approach to model your aggregates by presenting various approaches, their pros and cons, while breaking common mistakes from those new to DDD.
Editorial reviews by Deanna Chow, Liela Touré, & Pablo Martinez.
Want to work with us? Click here to see all open positions at SSENSE!
