Supercharging our event-sourcing capabilities with Phoenix LiveView
Building visual tools to tame event sourcing architecture complexity
In this article, I will describe how we have built some custom visual tools to reduce the complexity of working on WORMS, one of Casavo’s internal products built with the CQRS/ES paradigm.
Modeling and implementing core subdomains
One of the applications we work on in Casavo is WORMS (WORkflow Management System), which encapsulates the acquisition core subdomain. WORMS supports Casavo seller advisors and valuation analysts to track leads for interesting properties to acquire, evaluate them, estimate their profitability, and define an offer to the seller.
A core subdomain differentiates and gives a competitive advantage to a company over its competitors. By definition, core subdomains are complex and change frequently. The software for a core subdomain should withstand frequent changes to its business logic and provide good auditability and insights into the business processes it implements.
The inherent complexity of a core subdomain reflects in the software design. Advanced modeling and software architecture techniques are employed to ensure the solution is resilient to frequent changes over time. Two such techniques are Domain-Driven Design and Event Sourcing, which we used to implement WORMS. More specifically, WORMS is a web application built in Elixir on top of Phoenix and Commanded.
Commanded for CQRS/ES
Commanded is a convenient framework to model applications based on CQRS and Event Sourcing. Such architecture helps us implement complicated state transitions and business rules and protect business invariants. On the other side, an event-sourced application has a lot of components that dramatically increase complexity compared to traditional CRUD applications:
- Aggregates receive commands and emit domain events that are stored in the application event store;
- Workflows listen to emitted domain events to send commands to aggregates or communicate with external systems;
- Projectors listen to emitted domain events to update the application read models;
- Process managers coordinate complex business flows
How can we manage the complexity of the system?
Observability has always been a cornerstone characteristic of software systems built with Erlang and Elixir; Erlang provides the observer GUI to expose a lot of info about the running system; Phoenix takes this approach further by providing a built-in LiveDashboard. Even with these tools, getting insights into the status of a Commanded application can be pretty hard.
Inspired by the tooling of the Elixir ecosystem, we have built a few utility GUIs with Phoenix LiveView that help us tame the complexity of event sourcing and explore the state of the application:
- Event handlers status page
- Aggregate history debugger
- App schema visualizer
Event handlers status page
A lot of the behavior of the system is implemented through event handlers. Event handlers are processes that react to domain events to perform several tasks, such as interacting with external systems (e.g., email providers) and sending commands to aggregates.
Event handlers create a persistent subscription to the event store, keeping track of the last received and acknowledged event.
To have a quick overview of how event handlers behave, we built a LiveView page that lists all event handlers and shows the last acknowledged event for each one.
By computing the difference between the total number of emitted events and the offset of the last event seen by each event handler, we can color code each row to see at first glance which event handlers are “stuck” or are slow in processing events.
N.B: This is no substitute for having proper monitoring in place! Nevertheless, it is a handy tool.
Aggregate history debugger
Interactions with a Commanded application are initiated by sending a command to an aggregate. The aggregate handles the command by emitting one or more domain events and updating its internal state. An aggregate’s state can always be rebuilt from the ground up by reapplying past domain events. In event sourcing, the events act as the strongly consistent single source of truth for the application state.
Sometimes, we want to check which events have been emitted by a given aggregate. Doing so can be handy to diagnose bugs or to get insights into when specific events occurred.
While it would be possible to retrieve the list of events for an aggregate by querying the event store database, we wanted a more visual solution. To this end, we built the aggregate history debugger: the page displays the latest leads that have received an update in WORMS and provides a form to search for a given aggregate by its id. Doing so brings up a timeline of emitted events; each can be expanded to read its JSON payload.
The page reads the latest leads from the leads read model:
latest_updated_leads = Repo.all(
from l in Lead,
limit: 10,
order_by: [desc: :updated_at]
)For the aggregate search, we compute the id for the aggregates event stream and use the event store API to retrieve it:
stream_uuid = "#{aggregate_type}#{aggregate_id}"
{:ok, events} = EventStore.read_stream_forward(stream_uuid)The selected aggregate type and aggregate id are synchronized with the page URL query parameters to allow sharing the link for a specific aggregate search.
App schema visualizer
The app schema allows visualizing relationships between Commanded entities without browsing the codebase. It provides a combo box to select aggregates, commands, events, or event handlers and draws the appropriate graph.
Each entity has a different color coding (inspired by the EventStorming color coding), and each vertex in the graph can be clicked to inspect the code of the respective Elixir module.
Having this kind of visualization and exploration tool comes in handy when we need to review what event handlers listen to some particular event, or similar use cases.
The app schema graph is resolved at compile-time, by matching module name patterns in WORMS codebase and generating a JSON representation of the graph. The LiveView renders the appropriate graph portion for the selected node via a LiveView client hook that uses the vis-network npm package to draw the graph.
Conclusion
Event sourcing is an advanced architecture pattern best suited to implement a core subdomain complexity. Visual tools to visualize the system behavior and the relationships between its components help us tame the complexity as the system grows and evolves. Commanded and LiveView are great technologies that make building on top of their components easy.
Hopefully, we will soon be able to contribute to open source some of the solutions we implemented so far in our event sourcing journey!
Related reads
If you want to read more about WORMS:
- Business reliability as a cornerstone of our architectural choices
- From stair to lift: why we moved to a task-centric approach for our workflow management system
- Type-Safe API without writing and maintaining them in our CQRS + Event Sourcing Application
If you want to read more about Domain-Driven Design and CQRS/ES:
