View Models: Ephemeral Read Models
reSolve supports View Models, ephemeral Read Models created by reducers and maintained on server and client at once.
If you have created applications using the Event Sourcing pattern, or at least investigated it out of curiosity, you have come across the Read Model as a building block of Event Sourcing based applications. The term is not always used in exactly the same way. Some architects use it to describe a data structure used on the read side of a CQRS application, others apply it to the service which delivers such a data structure.
For the purposes of this article, a Read Model is a module of an application system. In an Event Sourcing system, events are published through a bus or a queue, for consumption by application modules. A Read Model receives such events and selectively executes Projection functions to persist parts of the information contained in the events. It chooses the structure of the persisted data freely, and it may execute calculations or aggregations as part of the process.
Read Models in reSolve
Using the reSolve framework, a simple Read Model projection function may look like this:
The projection function accesses the data storage system and persists information from the event, both metadata like the aggregate ID and the event timestamp, and process specific payload information.
The second important part of a Read Model is a Resolver. This is another function, which retrieves data from persistent storage and makes it available to a caller.
Sometimes Read Models have multiple Resolvers, but part of the Read Model concept is that queries should be simple. Data should be persisted in shapes that equal the requirements of client applications. If different shapes of the same data are needed, multiple differently structured storage collections can be used. Stored data is usually “flat”, not normalized in relational database terms. This approach means that queries don’t use joins or aggregations and data can be supplied as fast as possible. Of course there are no technical limitations and a programmer can persist and query data in any structure that offers the greatest benefits for a use case.
Note that each Read Model implemented for an application system can use its own dedicated storage. The examples above hint at a document-oriented database, but a Read Model can use specialized storage and retrieval mechanisms as needed.
One of the greatest advantages of Read Models is that any complex logic they encapsulate is executed asynchronously during event projection. For each event, a Read Model projection function can access existing persistent data, run calculations of any complexity, and generate as much new or modified data as necessary. The longer a projection takes, the greater the latency of the system, the delay between an incoming command and the eventually achieved state of consistency. However, at any point in time the Read Model data can be queried by clients with great efficiency.
When a client queries a Read Model, it receives a snapshot of data at that point in time. Generally, in order to receive changes, it needs to rerun the query. It is possible to implement a system of change notifications, so that clients are informed when changes occur and requeries of complete datasets are not necessary. We are investigating this approach for reSolve, but there will always be limitations due to the potentially complex processing a Read Model can perform during persistence. There is no prescribed logical connection between data that reaches a Read Model as part of an event, and data published by the Read Model through its Resolvers.
Read Models are built from the event stream of an application. If you ever need to change the logic implemented in projection functions, you can rebuild a Read Model by replaying events from the event log. This is a powerful aspect of an Event Sourcing application, because it allows you to fix issues or extend functionality without risk of data loss or limitations due to early planning mistakes. Of course it takes time to rebuild complex Read Models, more and more time as the event log grows.
View Models, or ephemeral Read Models
With all this background information out of the way, here’s an idea: we could construct a Read Model on the fly, just in time, when it is queried. This type of Read Model would not persist data and it would be dropped as soon as it has been sent back to the client. reSolve supports this kind of ephemeral Read Model, and it is called a View Model.
There are several advantages to this approach. First, since a View Model doesn’t persist data it doesn’t need to deal with persistence code. Instead, it needs to process a sequence of events when a query is executed, and generate a result data structure from the events. reSolve uses reducer functions for this purpose, similar to those used by Redux and other state management solutions, as well as by the React useReducer hook.
The code of this example reducer is only slightly different from that shown above for the Read Model projection. However, there is an important difference in its application: it does not depend on a storage system and it can be executed anywhere as a result. For instance, it can be executed directly on the client to apply data updates based on events, without further roundtrips to the server.
Consistency is the first advantage of the View Model. It is built when queried, there are no delays due to the latency of database persistence. No separate notification mechanisms are required for client-side updates, since event publication is already part of the Event Sourcing pattern.
The second important advantage is the minimal effort required to prepare a View Model. As a developer, you need to write only the reducer functions, and write them just once since they are used on both server and client. It is possible, and sometimes necessary, to write resolvers for View Models as well, but this is mainly required for security purposes. Generally, a View Model can supply data to a client without the need for a special interface function.
One potential issue with View Models is their efficiency. They are not inherently inefficient, but it is important to consider the implications of the on-the-fly construction for your use cases. Theoretically reSolve could project all events in the event log when a View Model is queried. Instead, it analyzes the View Model setup and projects only those events which are observed by the View Model. This is an important difference, but in a typical application the number of resulting events may still be very large.
For this reason, a general recommendation is to limit View Models to one or more specific aggregate IDs. The developer makes this choice in two places. First, the view model can either build a single object (like the sample above) or a list of objects. Second, when a query is executed, one or more ID values can be passed to restrict the events considered for View Model construction.
There are practical limits to the complexity you should implement in a View Model reducer. The logic is executed synchronously for the first query, delaying the response to the client. The response time of a View Model query depends on the number of relevant events in the system and on the performance of the reducers you supply.
It is possible to use snapshots with View Models. This means that reSolve automatically retains a persistent copy of the data when a View Model is queried, and reuses that snapshot if the same query is executed again in the future. Only events added to the log after the snapshot timestamp need to be projected when that second query runs. The benefit of this approach can be considerable, but the snapshot works only if the set of aggregate IDs is the same for both queries.
Finally, if a View Model represents a list of result objects, or if a single object includes lists of associated result data, it is important to consider whether all the information is used by the client. If parts of the data are filtered out, the efficiency of the initial construction and the transfer of the data may not be optimal.
When should I use View Models?
When you develop a new reSolve based application, it is generally a good idea to start with View Models for almost all data queries. They are easy to create and modify during development, so that application functionality can be implemented iteratively without much maintenance overhead.
During the development phase, you may find that some View Models evolve to use complex projections, or that external services are required or at least beneficial for a use case. You can then selectively shift towards Read Models for these scenarios.
Before you consider deployment, you should analyze and test your View Models with data volumes resembling production reality. At this point you may find that query performance suffers for some of your View Models. You can consider limiting them more than you did initially, or you may decide to convert them into Read Models.
Keep in mind the recommendation to limit View Models to a small number of aggregate IDs. In many scenarios where lists of data are displayed, a Read Model will be preferable in deployment. For “detail view” scenarios, where all data visible on screen is accessible using a known ID, View Models provide a powerful solution.
Whenever you decide to use a Read Model instead of a View Model, you will benefit from the structural understanding you gained during the development phase. Your Read Models will be optimized from the start and you won’t need to regenerate them too often, especially not in production.
In most applications, you will find numerous scenarios where View Models can be used efficiently in production. For the time being, they alone deliver the valuable feature of automatic client updates. Whenever you feel limited by the functionality they supply, the step to a Read Model is an easy one to take.
