Polyglot heaven — a response to the BEKK Radar — Bekk Consulting — Medium
Skrevet av Tomas Jansson
Last week I started writing on a blog series about CQRS and event sourcing using Event Store, Elasticsearch later on neo4j . When finished I realized how well the 10 posts were aligned with what we at BEKK recommends you should start using or at least evaluate.
The code for the sample application is on github: https://github.com/mastoj/CQRSShop, and an overview of all the posts for the blog series is available here: http://blog.tomasjansson.com/tag/cqrsshop/. The basic functionality in the application is summarized in the picture below:
In the BEKK radar which we updated about two months ago (see this post) we mention several techniques that make it easier for most developers to create and maintain complex applications. It was the drive to show that it is possible to tackle complex problem in a divide-and-conquer-like manner that made me want to share my experiences. I will not rephrase the whole blog series here, I will just relate it to the BEKK Radar. There are three higher level categories that relate to this article: architecture, programming and persistence. I will reference the numbers in the radar so you can find which subjects it relates to.
As a basis architecture for the application I’m using CQRS, or Command Query Responsibility Segregation. This pattern says you should separate your reads from your writes on an architectural level. For me this is just a divide and conquer algorithm applied to an architecture, and separating the reads from the writes enables the programmer to focus on one problem at a time and solve each one in the simplest way possible. The BEKK Radar don’t have a separate point for CQRS, but it relates to layered architecture, point 21. I think the application goes beyond a typical layered architecture, and instead layers (segregations) where you need them. On the write side you most likely will have more layers in a CQRS application than on the read side — since you have dedicated views for the reads and those can be mapped almost directly to the UI layer.
The application is also based on events (event driven, point 20). I’m using event sourcing for the domain, which means that I’m not storing the state for the domain, just the events that take place and have happened since the application was installed. If I want to know some state at some point in time I then run a projection over the relevant events — collapsing them into a representation of current state. Many people think that this is a source of performance issues, but it is actually not the case. When performing an action you will first need to look up all the events based on one id, this is an indexed operation which is really fast, and then you build up the relevant state so you can execute the action. The resulting events will then update a “static” read model asynchronously (asynchronous services, point 24) which is optimized for the type of querying you need. The update of the read models is done by services that could be regarded as micro services that has strictly one purpose.
Scaling the read side of this type of architecture is simple since you can scale horizontally, point 84, and since you will likely have a read-heavy traffic load the scaling problem itself is likely solved. If you need to scale the write side as well that is also possible with sharding or other techniques.
Even though I did not write the sample application for the JVM one of the subjects in the radar that fits with what I’ve done on the programming side is polyglot JVM, point 73. If you exchange the JVM with the .NET CLR this matches my application. I think polyglot programming is something that we will see more of on the server side, and we will definitely see more of functional programming languages like F#, point 47, which I have used in this application as well. I didn’t go all in with functional programming, I just used a language that in my opinion expressed commands and events better than an object oriented programming language. I think that I’ll try more of functional programming in the future since a functional approach fits very well with events, projections and immutability — all of which I’m using a lot in the sample application.
If you think polyglot will be big on the programming side, wait until you discover polyglot persistence, point 87. Using the right type of data store model for the job will make querying and storing data faster, easier to scale and easier to maintain in the long run. I personally thing using an event store (actually not mentioned in the radar, but fits under polyglot persistence) like the one from Event Store, is a great way of storing data in the case where you are interested in events that have happened. To query the data you use services that listen to events from the event store and then update the different read-side models. This can be done with a search engine/document database like Elasticsearch, point 38, or a graph database like neo4j. If you have some reporting problems that can’t be solved using any of these two it is possible to also update a classical relational database with data from the event.
I thought I should end this post with a short table of contents for the original blog series:
- Project structure — how the project is structured as a .NET solution
- Infratructure — the infrastructure is the thing you should write once and just once, and not create one repository for each domain object
- The goal — one step back to see where this all is going
- The first features — a more thorough step by step guide how the first features were implemented
- The rest of the features — the implementation of the rest of the features (even with tests)
- Time for reflection — a short mid way post about positive effects so far
- API implementation — an amazingly simple API implementation using Simple.Web instead of the standard .NET stack
- Integrating with elasticsearch — how all this can be integrated with Elasticsearch
- Integrating with neo4j — a bonus post where I integrate neo4j as well
- Ending discussion — my summary and some comments on the drawbacks of an architecture like this one
Originally published at open.bekk.no.