Event Sourcing explained with real code

Published in

Yellowme

5 min readDec 31, 2020

For the last 5 months, I've been learning about event-driven architectures and a pattern that is known as Event Sourcing. A funny thing about taking this path is that there are very good resources to learn about this topic on a conceptual level but there aren’t many examples with code that you can bring to a production environment with confidence.

So I think it's a great idea to just jump to the pool and start exploring some of the concepts of Event Sourcing with code written in a working environment.

For that, we'll write a Pokemon API that will let us encounter a pokemon in the wild with some randomly generated values (such as "attack", "defense" and "gender"). We'll explore the important parts of the pattern using Ruby on Rails and Rails Event Store.

What is Event Sourcing?

Event Sourcing is a backend architectural pattern that stores the domain changes on the application as a series of replayable events.

I don't want to go too deep on why Event Sourcing is good or what other side effects you can get with it because I think that there are very good articles about that already. But some of the main reasons to use this pattern are:

You'll have the ability to destroy and re-build the entire database only replaying the event history
You'll have an event queue that avoids database conflicts
You'll be able to separate domain errors from system errors out-of-the-box
You'll separate the concerns in a domain-centric way

There are a couple of important parts of the pattern with different purposes:

The events will define what happened in the domain
The aggregates will handle the business rules
The projections will persist the data to make better queries

Also, we'll explore how to:

Handle the different type of errors
Write tests with this pattern

Events

Since we're taking an event-driven approach with the domain, we should think of events as something that happens on the domain at a given time.

So we need to declare a simple class that can carry our event data.

Note that the WildPokemonAppeared class inherits from RailsEventStore::Event from the Rails Event Store gem and will help us to store, apply and subscribe to that event.

To store and apply an event we have to create an aggregate.

Aggregates

An aggregate is like a model but with a state.

That means that it’s the class that will define the domain and has the responsibility to:

Enqueue events to the event stream
Replay the event stream when instantiated
Handle entity state on memory
Decide if an event can happen based on that state

That's a lot, right? But don't worry too much because most of these things will happen behind scenes and will become pretty obvious once we get used to it.

Here's an example of a Pokemon aggregate:

There's a couple of things happening in this aggregate:

We're including the AggregateRoot from the gem to have the apply method that will store the event and send it to the stream.

And thanks to that module, that apply_wild_pokemon_appreared method will be triggered once the Events::Pokemons::WildPokemonAppeared is successfully applied.

Now without an explicit pokemons database table, we can persist the data of a pokemon.

And we can also query the event directly from the event store:

Projections

While using aggregate methods to write data is ok, most of the time we'll end up reading that data.

For example: What would you need to do with this pattern if you want to get a collection of all the pokemon with a level greater than 50? You'll have to filter all the reconstructed states of all the Pokemons. And that's crazy expensive for the server.

Instead of doing that we're going to make use of projections or calculators.

A projection consists of persisting the last state of an aggregate outside of the event store.

Here's an example of a projection for the WildPokemonAppeared event:

Note that I'm using the find_or_initialize_by method instead of new because we want our projections to be idempotent. That means that every time we trigger the same event with the same data we should get the same result or projection. This is something to keep in mind when using this pattern.

Oh. And one more thing: We have to subscribe the projection to that event explicitly:

With this, every time the WildPokemonAppeared event is applied, the OnWildPokemonAppeared projection is going to be triggered.

Errors

There are mainly two types of errors that we will have on an event sourcing flow:

Domain errors
Projection errors

The domain errors are the exceptions that you throw on the aggregate that will prevent the domain to do unwanted changes to the state. These errors should prevent the events from being applied.

An example of this is the InvalidPokemonStatus error from the Pokemon aggregate. And here we’re forcing the domain to throw that exception just by calling the appear_in_the_wild method twice:

Because remember that on the Pokemon aggregate there’s a validation to prevent this method from being called twice:

Another common type of error found on event-sourced systems is the projection error that consists of unexpected exceptions that are thrown on the projections. For example, a database error or typo could cause that a projection is not reflected properly:

The interesting thing about this type of error is that we can fix and replay the event projection without any interaction from a user at all.

And in code you simply have to do something like this:

This is a side effect of storing an event every time something happens on the domain. And also having on a different module all the projections that those events cause. We could literally destroy the entire Pokemon table and replay all the Pokemon events to rebuild it exactly as it was before.

Tests

Since Event Sourcing makes use of many classes that interact in a not-so-traditional way, the best approach I found yet is to test the most critical parts of the domain with a unit test on every aggregate method.

Here’s an example of a unit test of the appear_in_the_wild method from the Pokemon aggregate:

We’re using RubyEventStore::InMemoryRepository to avoid storing every event on the database during the execution of tests. That will make our tests run faster.

And it’s important to add a projection test too.

Remember that testing idempotency is something that you should care about when writing the projections. That’s what the second test is trying to achieve by calling the projection twice.

Is that it?

Well, yes. But actually no.
If you’re reading this, that means that you should now know a little bit about the actual implementation of Event Sourcing and not only the wonderful concept of it.
There are a few things that I think you should know to take advantage of this pattern on a production application.

How to validate types?
How to validate user roles and policies?
What is CQRS and why its good for Event Sourcing?

So in my next post, I’ll be sharing more code with you to illustrate the CQRS pattern and other solutions that will make sense if you're also on the path to understanding event-driven.

You can learn here about why our team at FirstRoot is building the world’s first platform for democratic financial decision making in schools. Also, you could expect more posts in the future about why it's important for these types of systems to have an event-driven architecture with full traceability.

Here are some of the resources I read and could be useful for you:

Event sourcing basics

Should I use it?

And finally, all the code showed here is on Github.