Serverless, DDD & Vertical Slices
Serverless is becoming an ever more popular choice for cloud development. But what is it? Why would you choose it? And how do you get the most out of it? In this article, we shall aim to answer these questions. Firstly by looking at it through the lens of Domain-driven design (DDD). Secondly by applying vertical slice architecture to decompose a complex problem.
What is Serverless?
Serverless is a method of on-demand cloud computing where resources exist transiently only when they are in use. It is closely associated with the execution model of Function as a Service (FaaS) which has been popularised by offerings such as AWS Lambda and Azure Functions. Serverless extends well beyond compute resources though with an entire ecosystem of operational resources. There are still servers, but their management and maintenance are abstracted from the developer.
Why Serverless?
In his seminal book “Patterns of Enterprise Application Architecture”, Martin Fowler stated that “the first rule of distributed objects is don’t distribute your objects”.
Distribution opens a pandora’s box of complexity. The increased cognitive load and unpredictability of asynchrony. The need to consider failure handling, consistency and availability. For sure, when scaling distribution can become an architectural enabler. You can trade off that complexity cost against scaling development, scaling production usage, and increasing resilience or performance. But start off with a monolith. Keep it simple.
So with this in mind why on earth would you start off using serverless functions which are distributed by default? What are we trading off? In a way, it is simple economics.
Continuous Delivery
“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software” Agile manifesto
The modern practice of Continuous Delivery was named after this statement. It captures the essence of the Lean/Agile promise for software development. The ability to maximise ongoing ROI by continually delivering value. Creating immediate returns through customer usage. Enabling constant learning of market needs and demands. Driving efficiency, quality and development focus by the need to keep transaction costs low.
But it is one thing to develop a product. An altogether different challenge to deploy and operationally manage it. This distinction has traditionally led to a divide between development and operational IT. The problem with having different teams across a value stream is that it creates a handoff. This inhibits flow and without flow, you cannot be continuous. The DevOps movement aimed to break these silos down and reestablish flow but in many cases, this has merely led to a rebranding of the operational IT team to the “DevOps” team.
Serverless can re-enable this. Firstly it removes almost all friction from the technical process of deployment. Secondly, it commoditises many of the concerns associated with operational management such as server provisioning and maintenance, patching and scaling. This reduction in specialisation and cognitive load creates the prospect of having a single team to develop and deploy a business capability or product. One that is freed from what AWS CTO Werner Vogels calls the “undifferentiated heavy lifting”. Thus allowing a focus on software development that differentiates their own product. A focus on their own business purpose.
Pay-as-you-Use
“If a poor workman was obliged to purchase a month’s or six months’ provisions at a time, a great part of the stock which he employs as a capital in the instruments of his trade, or in the furniture of his shop, and which yields him a revenue, he would be forced to place in that part of his stock which is reserved for immediate consumption, and which yields him no revenue. Nothing can be more convenient for such a person than to be able to purchase his subsistence from day to day, or even from hour to hour, as he wants it. He is thereby enabled to employ almost his whole stock as a capital. He is thus enabled to furnish work to a greater value” Adam Smith, 1776
Another Serverless enabler for better ROI is pay-as-you-use for operational resources. This is particularly useful at the startup level. Where the challenge is not only to acquire outside investment to pay for the development of your solution but the upfront infrastructural cost to get it into production. Even at the enterprise level, any new business proposition is going to be easier to sell when it needs less upfront investment.
This is also very useful during development. Without Serverless, creating a production-like infrastructure for testing can become very costly. It often leads to trade-offs with the number of test environments and how production-like they really are. But with Serverless you can spin up multiple test environments with next to no overhead. Only incurring costs as they add value when they are used.
So Serverless has economic promise. But will that not all be undone by the development cost of dealing with the increased complexity of distribution? How do you turn a pile of independent transient functions into a coherent product?
Domain-Driven Design (DDD)
There is a layman’s interpretation of DDD that it is no more than a handful of software design patterns. It is way more than that. Think of it as a user guide to effective software product development, from organisational design to individual use cases. It is no coincidence that at the heart of many of the most popular ideas of modern software development, you will find the influence of DDD. In Team Topologies, Microservices, Event-Driven Architectures & NoSQL databases.
DDD is often separated into strategic & tactical design. But for the purpose of describing its usefulness for Serverless, we will further divide it into four scales:
Complex Product Decomposition — whisper it quietly but there is no such thing as an autonomous software team in a large product. At least in the purest sense. You cannot entirely isolate a part from the whole. What it really means is a team that can make enough of its own decisions to effectively develop and deliver some business capability. The key is to architect for minimal coordination with other teams. DDD helps in this regard by providing ways to decompose a large product into loosely coupled business capabilities (e.g. through event storming). It also identifies a set of patterns so that any remaining coordination between teams is categorised into a relationship pattern so that expectations are clearly set on cross-team engagement (e.g. partnership, conformist, open host). Additionally, business capabilities are classified (i.e. core, supporting, generic domains) to provide strategic clarity on decisions on how to invest and resource their development. So that overall business value is optimised.
Business Capability — the start of any human collaboration is a shared language. In business, we create solutions to problems. DDD immerses software teams in the language of the problem domain, of their customers and markets, and their desires and needs. Also, it recognises the messiness of language. Look up any word in the dictionary and it has many meanings depending on context. As a solution becomes complex, different users or roles emerge with different needs and terminologies. The set of language to describe the product grows, diverges, overlaps and ambiguities emerge. Within the bounded context of one of these perspectives, a subdomain, the language remains specific and unambiguous. Seen from the perspective of a specific actor, role or domain specialism. That specificity keeps the language ubiquitous within that context. Finding boundaries of business capabilities on this basis allows teams to strongly own and connect with their business purpose.
Domain Object — this is the level that contains the software design patterns. But these patterns are merely symptomatic of a more holistic business-oriented objective of modelling long-lived business concerns. From a conceptual level, it recognises numerous characteristics of how things function and exist in the real world. Such as creation (i.e. factory). Unique identity (entity). Constraints on how something exists (value object). Existence over a lifetime (and therefore the technical constraint of realising that through persistence i.e. repository). Evolution of current context (like a state machine) and complex existence made up of other things (aggregate). One of the biggest dysfunctions of microservices adoption is trying to decouple functionality into services without decoupling the database. DDD provides a way to do this decoupling whilst ensuring you retain the consistency of data you normally associate with a centralised relational database. Think of it as object-oriented development done well. With strong cohesion of data and behaviour and tight encapsulation (rich domain model) to maximise loose coupling.
User Interaction — this is the most granular level of customer impact. Where someone or some other system interacts with the system. The nature of an individual interaction falls into two categories. A command where they are attempting to change the state of the system (of a domain object). And a query where they are learning something about the current state of the system. The primary concern at this scale is ensuring that the domain objects with the associated business rules and constraints are kept isolated from the technical plumbing that allows interaction with the user (driver ports) and access to infrastructure such as persistence, messaging, etc (driven ports).
Defining a Serverless function
So what is a function modelled upon? Let’s consider our scales of DDD.
Business Capability? This is the scale normally used for defining a microservice. What DDD refers to as the bounded context. It is really aimed at a linguistic boundary for a team. A function is much smaller than a team boundary.
Domain Object? This could be used but it is long-lived. This seems out of sorts with the transient asynchronous nature of a function.
User Interaction? Small. Specific. Purposeful. Transient. Now, this seems like a good fit. We also need a degree of isolation in our code structure as the function is deployed independently. Vertical Slice Architecture is an approach that lets us do exactly that. Initially described by Jimmy Bogard, it has been championed by others in the .NET community such as the excellent Derek Comartin.
Kicking & Screaming Architecture
Software development is primarily associated with changing code. But in reality, more of our time is spent reading than writing code. Constantly traversing folders and files to orient, navigate and interpret.
When defining Clean architecture, Bob Martin highlighted the importance of the software structure screaming the business purpose it represents. Rather than an over-focus on technical concerns such as frameworks. In Clean and Hexagonal this is achieved by conforming to clear separation of concerns.
A set of horizontally sliced layers through the code. Each is represented by a folder. This could include business-oriented folders such as “use case” or “domain”. Alongside more technical folders such as “infrastructure” or “UI”.
Given that the primary axis of change in software is the business purpose and that the most granular impact to this is a user interacting with the system, a lot of the time change winds up involving updating or creating a user interaction.
With horizontal slices, this often leads to changing or adding little pieces into each layer. Something that requires constant traversing back and forth. It is this lack of cohesion that led Jimmy Bogard to the premise of vertical slice:
“Minimize coupling between slices, and maximize coupling in a slice.”
Horizontal slices are orthogonal to the primary axis of change. The vertical slice approach corrects that.
It starts with a features folder containing a subfolder named after each feature (i.e. a domain object or a CRUD data model). In each feature subfolder, you will find a file named after each user interaction on the feature.
When you change or add a user interaction you can do it all in one file.
At the heart of each file will be a command or query handler. This singular focus on a command or query means vertical slice embraces the CQRS design pattern out of the box (although not necessarily event sourcing). This means that a domain object and persistence mechanism such as a repository can be focussed purely as the write model. On evolving a domain object. Helping to avoid dysfunctions such as anaemic domain models or bloated repositories.
Queries are free to access the data however they like. They might use direct access. Or be optimised through a projection.
Keeping It Dead Simple
“Simplicity – the art of maximizing the amount of work not done – is essential” Agile Manifesto
Simplicity is so important in software development. It comes down to a fundamental limit on how we interpret our world. Basically, complexity hurts our heads. This is nicely explained by cognitive load – the limit of our working memory. It underpins many of the preferred ways of working we see in software development. Like working in small teams. Code abstractions such as modules, classes and functions. Doing the simplest thing in the green step of test-driven development. Continually merging small increments in continuous integration. Working with WIP limits.
Vertical slice reduces cognitive load by lessening the traversal of folders and files. But another way it does this is in its approach to abstractions.
Clean/Hexagonal uses dependency inversion and a strict direction of dependencies within layers to ensure that business concerns are kept separate from technical concerns. This keeps the business policies clear. But it also allows flexibility of technology change which generally evolves quicker than the business policies.
“The outermost circle consists of low-level concrete details. As you move inward, the software grows more abstract and encapsulates higher-level policies.” Bob Martin
Vertical slices do not start with all the layers and abstractions. After all, they are protecting against complexity which does not yet exist within the confines of a newly defined user interaction. Would you really add abstractions to a hello world program? It merely forms code clutter.
So YAGNI.
Instead within the context of each user interaction continually refactor and introduce the abstractions if the requisite complexity emerges. So start with CRUD and direct access. As duplication emerges abstract your data model. As more complexity emerges transition to a domain model.
Simplest serverless
Using the vertical slice approach in Serverless a function represents a user interaction. Having most of the code in one file means it is ideally isolated to be packaged and deployed. That would not be so easy using traditional hexagonal DDD where you have an application service which covers all interactions relating to a domain object. Or Clean Architecture where you have a use case with a set of interactions relating to a business goal.
Of course, functions will also be needed to support underlying infrastructural needs. Such as the outbox pattern to ensure consistency between domain object persistence and event publishing. Or projection handlers to support queries.
And taking a step back how do all these transient functions come together into a coherent whole?
Partly through commands and queries that are driven by external user interactions. Partly via the persisted domain objects that they interact with. But Serverless functions are inherently event-driven which leads us to yet another reason why DDD is such a good fit for Serverless.
Whenever a domain object is changed, we can generate a domain event, another key concept from DDD. These events can be used to trigger the invocation of other functions. They can cross team boundaries into another bounded context making them integration events. Part of the loose coupling between teams. That can then be marshalled through techniques such as consumer-driven contract testing.
Conclusion
Simplicity demands we reason in the small whilst being connected in the large. So try combining Serverless with DDD and vertical slices. Allowing independent transient functions to weave together into a coherent product. Whilst enabling the human considerations of cognitive load, collaboration and purpose.
