The journey to Domain-Driven Rails and TDD
I have always struggled at deciding whether it’s better to write a short/elegant code or it’s OK to have a bit more verbose code, but a code which is more intentional.
Often, when you read about such dilemmas, you will see two ad-absurdum arguments. The developers who prefer shorter code, will show some kind of GenericFactoryBeanFactoryServiceImpl.java to prove that the so-called “architecture” astronauts are just plain stupid.
On the other hand, we have developers who like to have some classes/objects and they go ad-absurdum from another direction, showing some Perl-like one-liners, which clearly “prove” that it’s just wrong to have too short code.
The problem with such arguments it’s just that they focus on two extremes, while there’s a huge grey area and it’s not a binary choice.
If you follow TDD with just the Red/Green cycle (just focus on make the tests green), you will end up closer to the one-liners. If you focus on the Refactor phase for too long, you may end up with some complicated and unclear abstractions.
My love to TDD was powered up by the Red/Green/Refactor rule.
What should happen in the Refactor phase? Should the “just enough” code be extended? Is there any rule I can follow here? How far should I go with abstractions?
For years I’ve relied on intuition here. It was OK to keep my TDD flow going, but I’ve always had the feeling that something is missing here.
At some point, I’ve learnt more and more about DDD. In its core DDD has not much to do with TDD. It’s just a set of guidelines how to make your code more domain-oriented. However, I liked it so much that the DDD patterns (both strategic and tactical) became favourite techniques in my toolbox.
DDD was the missing piece in my TDD flow.
DDD answered me the question — how far should I go with abstractions.
The thing is it’s not just about abstractions. It’s about our understanding of the domain. How do we think about the domain of the problem? How do we make a model out of it? How do we represent the model in our code?
DDD helped me find the balance between the always-tempting primitive obsession (trying to map all domain problems to integers, strings, hashes and lists, in Rails you can see it as solving everything with tables/associations/callbacks) and the world of abstractions which no one can understand.
I’m not going to defend artificial abstractions here. However, I’m going to favour abstractions which represent the actual domain world. If I’m working on some Genetics software, I want to have a DNA or a Strand class instead of a String.
I want my code to tell the story about the domain. I want the code to be as explicit about the domain as possible. I expect other developers to be able to learn about the domain from my code.
I’ve been practicing both TDD and DDD for enough time, that one day something “clicked”. I couldn’t do one without the other. DDD became the toolbox for the Refactor phase in the TDD cycle. Now it was more clear what I’m meant to do. I’m meant to reflect my domain knowledge in the code. The tests are passing, the requirements are implemented, but now I can shape the design in a way which reflects my domain knowledge. I’ve started to follow Domain-Driven Development.
Even though, it was easy for me to apply TDD+DDD in my side projects, it wasn’t so easy to do so in the Rails apps, which happen to be our main kind of projects at Arkency. To be more generic, this combo was hard to apply within any framework-infected code. Rails just happened to be an example of it.
This triggered my research how to solve the problem of Rails apps and writing clean code. It took me several years but I found a bunch of techniques (mostly from DDD tactical patterns) which help decouple our actual application from the Rails application. My research was heavily influenced by DDD and by Clean Architecture (by Uncle Bob).
In short, if we apply service objects, queries, repositories, form objects(known in other worlds as commands) and adapters we can split a typical Rails app into two parts — the Rails part and the actual app.
The Rails part would have the controllers call the services objects or queries.The models (ActiveRecord) would just become an implementation detail of the repositories. The params were wrapped with form objects. Everything that was below the service objects (or we can call them application service layer), but above the repositories would become the domain layer.
That was a huge milestone for me. I’ve had a repeatable and reliable way of extracting the domain part of my Rails app. In a way I could treat the domain as a library that is used by the Rails app. It can be in the form of a gem, but doesn’t have to be a gem, that’s just a detail.
It wasn’t just a dream of some kind of an architecture astronaut. In practice, being able to reliably make the split enabled me to do the TDD flow while working in my Rails apps, especially in the domain part. I could feel like doing those “non-practical”/unrealistist coding kata, while actually implementing important features.
Together with the Arkency team, we’ve been sharing our lessons from that process in the form of blog posts. At some point, it became clear that a blog is not enough and we’ve put the reliable process into a book called: Fearless refactoring: Rails Controllers.
I know that I’m not very neutral when it comes to recommending that book. However, over the last years it was a joy to watch the readers succeeding in applying the techniques from the book.
That book wasn’t the end of my struggle. What we showed in the book was a process of decoupling the domain from Rails. What we haven’t shown was the other two important parts. One of the parts is related to actually splitting the Rails app into 2 parts: the frontend and the backend (in most Rails app they become the same thing). The other part is about splitting the domain itself.
The first part is how to extract the frontend part of the Rails application. I’m lucky to be surrounded by a team of fantastic developers at Arkency who did the job of solving this frontend problem and (more importantly!) describing it. For years, we’ve struggled with it, but had one intuition — the frontend is a separate application. The revolution started to happen when 2 things appeared — React.js and the rise of the modern JavaScript setup (including webpack and Redux). We’ve explained our transition in two books: Rails meets React.js(targeted to Rails devs) and React.js by Example (targeted to anyone who is just starting with React.js and the modern JS setup — there’s 14 working repositories included!).
Just to summarise the story so far — we’ve managed to find a reliable process of decoupling the Rails app from the domain and we found a consistent process of extracting the frontend parts of our Rails apps into the JS world.
The journey has been incredible so far and I’m proud of what we’ve achieved. We’re not only able to enjoy (and be proud!) of our code while working on new Rails apps. We have also been able to transition from our older Rails appsinto this proud-giving state of code.
Are we at the end of the journey?
Far from it :)
I’m quite happy that whenever we achieve one milestone we already see other places which are worth our focus. The most recent (for about 2 years now) area of our focus was the problem of splitting the domain part.
For years, I was dreaming about this one model, one design that rules all of my application. A model which would be free of the infrastructure parts like http, database or other API — a pure domain. Learning about DDD taught me a lesson here — I wasn’t looking for one model. What I was looking for was a design which consists of several of such pure models. The answer I was looking for was called Bounded Contexts.
If you looked at the Arkency internal communications you would see the term Bounded Context (or BC as we often call it) growing popularity. In all of our projects we started to learn what are the bounded contexts — what is the ideal split of the domain. Sometimes that goal stays only conceptual for a long time, but at least we know what’s the target.
Our goal is to reflect the bounded contexts in our codebases.
A bounded context can be defined as this area of your business where you use the same business language or as they call it in DDD, where you use the ubiquitous language. Bounded contexts are often represented as departments in bigger enterprise companies. If you work on an e-commerce app, very likely you’ll have Inventory, Pricing, Promotions, Shipping, Invoicing, Payments, Reporting as the bounded contexts. Even though, they’re parts of the same system, you’re actually using different meanings for the same term.
In this e-commerce app, the concept of Product is different in all of those parts. In the Inventory world, a Product is something that exists in the store, there’s probably some amount/quantity value related to it. The availability is becoming an important concept here.
In the Pricing world, we’re mostly interested in the price of the product. Here, a product is something that has a connected price to it (or multiple prices). If you were involved in preparing prices for any products, you know how big this area is and how many dynamic variables need to be taken into account. All of that makes the Pricing bounded context.
In the Shipping world, a product is something that is part of the shipping process — taking the product from point A (usually the store) and deliver it to the point B (usually the Client address).
As you see, whenever we see a different meaning for a similar concept or whenever we see some groups of related terms, there’s probably a bounded context to be extracted.
In the codebase, the lack of bounded contexts is mostly visible and painful in the tests. If you ever had to prepare a User and their Password (clearly from the Authentication world), just to be able to test a Product object or the Shipping process, you know how frustrating it is. An example — in order to have a proper Shipping object, you need to have an Order, so first you need to have a Cart, so first you need to have a Product added by an admin, who needs to have the login/password. You also need to have the Customer object with a valid address object. That’s just a huge set of dependencies.
Knowing how to split your domain into bounded contexts is just one step in the process. Another important part is to allow those bounded contexts to communicate. This is where domain events come in. However, they’re not the only way. We can simply have an Application Service layer which coordinates the bounded contexts.
So, to summarise this message:
- TDD is a great technique, but may leave you uncertain what exactly to do with the Refactor phase
- DDD is a good fit here, as it tells us to refactor to the point where our domain understanding is represented in the codebase
- Rails apps are hard to TDD, so we need to split them
- Separate the Rails app from the domains part by applying techniques from Rails Refactoring: service objects, form objects, repositories, adapters
- Separate the frontend from the Rails app, by extracting a React.js based JavaScript application
- What is left from the Rails app is the domain, but we need to split it into bounded context to easily be able to do TDD
- Enjoy TDD with Rails! :)
Psssst, I have decided to prepare a video class on the topic of TDD and Ruby/Rails. The class covers topics like the ones mentioned here — using DDD and also mutation testing to build highly modular, TDD’able Rails applications. If you’re interested in getting a 70% discount and have an EARLY ACCESS to the class, use this link (before Thursday, 19.05.2016 11pm CET): https://vimeo.com/r/1I82/STcxRjFuel
