Managed by Q helps people run offices. We have to be good at a ton of things, from filling administrative staffing openings to ensuring that office spaces are clean and well-maintained. We are currently evolving some of our core systems to better handle the variety of problems that we solve for our clients.
I’ve used this occasion to revisit Domain-Driven Design (DDD) by Eric Evans and, in particular, the Aggregate pattern. There are lots of descriptions online of what an Aggregate is and how to use one. But I’ve found it harder to explain why they should be used and to know where we should be using the pattern. I wrote this article to help myself get clear on what Aggregates are and when to use them. I hope it’s helpful for others as well.
Defining DDD and Aggregates
What is Domain-Driven Design? The main idea is for engineers and domain experts to agree on a “ubiquitous language” that describes the problem space in a way that makes sense to non-technical folks. That language should then dictate the design of software systems. When done right, the synergy between your problem domain and your software design makes it easier to build software that extends to fit future requirements.
An Aggregate is a specific software design pattern within DDD. NoSQL Distilled by Pramod Sadalage and Martin Fowler defines an Aggregate as a “collection of related objects that we wish to treat as a unit.” In particular, an Aggregate is a tree of objects. Like all trees, it has a root and a boundary.
Because Aggregates have boundaries, it’s easy to distinguish between public and private elements of the Aggregate; the root is the only public member of the Aggregate. That is, it’s the only member that other Aggregates can directly reference. Only elements within an Aggregate can hold references to non-root (private) elements of that Aggregate. For example, consider an invoicing Aggregate, whose root is the invoice. Line items may be children of the invoice. No other Aggregate may hold a direct reference to an invoice line item. They may only refer to the invoice root itself.
The boundary of an Aggregate also helps define a region of consistency. Domain-Driven Design states,
Invariants, which are consistency rules that must be maintained whenever data changes, will involve relationships between members of the Aggregate. Any rule that spans Aggregates will not be expected to be up-to-date at all times. Through event processing, batch processing, or other update mechanisms, other dependencies can be resolved within some specified time. But the invariants applied within an Aggregate will be enforced with the completion of each transaction.
So, for our invoicing example, the total of the invoice ($80 above) must always be consistent with the sum of the line item values ($40, $30, and $10). This invariant must always be satisfied; those values can’t even momentarily be inconsistent.
By contrast, state between Aggregates is eventually consistent. Let’s say that our invoice is generated when someone places an order. The Aggregate for the order may transition to the “invoiced” state immediately when the “buy” button is clicked. But the invoice Aggregate will be generated later — perhaps by milliseconds or perhaps by hours.
An example: Social graph
Let’s consider an example: a social graph of the form that Twitter uses. A person may follow another person. Following is a one way relationship. So if Jason follows Gloria, that does not mean that Gloria follows Jason. We shall also track the follower count and the followee count for each person.
There is a straightforward model for a social graph. We can have a
Person object that has a
follow() operation. In a relational database, we might implement this
Person with two tables:
Follow. Each row of the
Follow table would contain from and to columns, both of which are foreign key references to the
For example, let’s have Jason and Gloria start with no followers and no followees.
When Jason follows Gloria, we insert a row into the
Follow table. Let’s assume for the sake of simplicity that follower and followee counts are dynamically calculated by querying the
Note that, in this design, each
Person is not an Aggregate. A
Person doesn’t have a consistency boundary. One transaction modifies two
Persons. In the example, we modified both Jason’s and Gloria’s
Person Aggregates in one step. A
Person isn’t the root of a tree. It’s a node in a graph.
But suppose we continue with this non-Aggregate design anyway and we want to cache the follower and followee counts so that we don’t have to continuously query the
Follow table. We can end up with tricky locking behavior.
If we are pessimistically locking, we will need to lock both Jason’s and Gloria’s
Person objects. To avoid deadlocks, we would perform three separate transactions:
- Insert a
- Lock Jason and update his followee count
- Lock Gloria and update her follower count
If we are optimistically locking (possibly by using a strong transaction isolation level), we could do all three operations in one transaction. But performance may be poor for frequently followed
Social graph: Aggregate design
We can model the social graph with Aggregates by separating followership from followeeship.
Now, Jason’s and Gloria’s
Person objects are both Aggregates. They contain private
Followee objects that contain references to other
Person objects. Think of these references simply as raw IDs and not as foreign key constraints. On a database level, we would no longer have a
Follow table that simultaneously represents follower and followee relationships. Those relationships would be captured in separate tables.
But because there are two different aggregates, we need two different database transactions when Jason follows Gloria:
- We add a
Followee, referencing Gloria, to Jason and update Jason’s followee count.
- We add a
Follower, referencing Jason, to Gloria and update Gloria’s follower count.
So, Jason and Gloria will be eventually consistent with each other. To ensure that step 2 always happens after step 1, we would schedule step 2 during step 1’s transaction, likely by recording an event that we’d process asynchronously.
This pattern has clear locking behavior, regardless of whether we cache the follower and followee counts. We can lock Jason to add a followee and to update his followee count. We can lock Gloria to add a follower and to update her follower count. These two locks are completely independent of each other.
So what’s the point?
The Aggregate pattern imposes some constraints. Relational databases, programming languages, and popular web frameworks allow us to create arbitrary graphs of objects. But the Aggregate pattern requires that every object belongs to a tree and it limits each transaction to work on one Aggregate. Those requirements reduce flexibility and create work. Why would we agree to them?
The example above helps me think of a few reasons:
- Aggregates make it easier to reason about large systems. In the absence of Aggregates, a design that starts as a relatively simple relational graph can evolve into a sprawl. Aggregates have inherent boundaries that prevent sprawls from forming. That makes it easier to define ownership over elements, to extract separate services, and to build database queries that are easy to understand.
- Rules for maintaining invariants are clear. There are no ad hoc transactions or bespoke locks. Transactions are scoped to Aggregates and we lock the root of the Aggregate.
- Scaling is easier. Should our social network explode in popularity, we might want to place Jason’s and Gloria’s
Personobjects on separate database shards to distribute load. This approach isn’t practically possible with the non-Aggregate design, because both Jason and Gloria share a common row of the
Followtable. The Aggregate design makes sharding straightforward.
These benefits may not be worthwhile for small systems and experimental projects. But as systems get larger, the scalability and clarity benefits of Aggregates justify the extra work.
I hope this article helps clarify what Aggregates are and why they are used. I welcome pointers and links to other, better articles on this topic.
If you’re serious about finding ways to create ownership boundaries for service extraction, it may help to think at a higher level about the boundaries in your domain. I recommend reading about DDD’s strategic concepts, which include subdomains and bounded contexts. I find these concepts difficult to understand, and I might write a post on them one day.
I found these books especially useful in understanding Aggregates:
Thanks to John Chapin, Blake Dickstein, Matt Madurski, Travis Thieman, and David Van Couvering for reviewing drafts of this article and for helping me make it better.