The 7 Layers of Microservice Design
Article by Weave, written by Kas Buunk
Part 1 of 3 articles
“Microservices can buy you options”, is a well-put phrase that communicates two things:
- If you choose a microservice architecture, solutions arise for problems that are harder to tackle with a monolithic application architecture.
- There’s a cost to microservices.
This article illustrates how we consider and implement the microservice architecture pattern at Weave. I will explain what microservices are, state the main benefits and costs and focus largely on the role they play in our projects’ architectures. In future articles, I mention various other patterns and Go-specific solutions we’ve implemented to give meaning to the principles of good design.
Benefits
When done right, microservices offer benefits by independent:
- deployability: services can be released in the application infrastructure as separate containerised processes.
- development: teams can be aligned to own and be responsible for a set of services, whilst the service’s interface can be part of a contract towards other services and their development teams. This reduces the detail of the entire application that a developer needs to understand to just the non-owned services’ interfaces.
- scalability: horizontal scaling of just the microservice requires fewer resources. Performance-critical processes can be scaled up, without requiring memory copies of the entire program.
- failure: if one service’s process crashes, it does not interfere with the application’s other processes.
The best summary of the advantages is: separation of concerns. Done right, this increases maintainability, readability, modularity, and many more “*ilities”.
Costs
A microservice architecture inherits the complexity that arises with distributed systems. There are many things one ought to consider, and the number of pitfalls of bad design increases, as a result of a larger surface of communicating parts and interfaces.
The best summary of the disadvantages is: microservice architectures are risky and expensive, especially when chosen prematurely. If the separation of concerns is not done right, it is now a lot more expensive to refactor. If the coupling and cohesion are off, domain behaviour that is essentially closely connected may be distributed over microservices, requiring choreography over the network. Failure to get this right and handle errors can result in various problems. The network boundary makes it more expensive to refactor. Services can become entrenched into a position that requires a lot of effort to change, especially if multiple teams need to coordinate.
Weighing costs and benefits
The quantification of cost and benefits largely come down to the question: is it done right? Were the microservice boundaries chosen in alignment with domain boundaries? If requirements or insight change, will that likely require reflection in a small or large set of services and teams?
Scope at the service level
The point is made that choosing the right responsibility for every microservice is essential in applying the architecture right, reaping the benefits and mitigating the costs. This article will not go into further detail how this can be achieved at the application level. The discussion focuses on the microservice level, assuming a good application design, which deserves a separate discussion. That means: in the application as a whole, we assume the microservice is centred around its own, well-defined domain, fulfilling its role in the application.
Contracts and bounded contexts
The application should be easier to grasp by composing understanding of the microservices, as building blocks of the application. We assume each microservice has a proper bounded context, a term coined by Eric Evans in his book Domain-Driven Design. This implies the services communicate only through predefined, agreed-upon contracts: interfaces that unambiguously define schemas of what messages, communication protocol and serialisation format are accepted.
Services may not cross the boundary, i.e. they cannot depend on the inner workings of how another service fulfills its contract. The reason is straightforward: it would violate separation of concerns. This should be a clear guideline and be a topic for code review. Ideally, this should be guaranteed, which will be touched on below.
Service design
With that out of the way, we narrow down the discussion to the microservice level. This section describes the layers, patterns, principles and goals of a good service design. For now, we leave abstract how we implement this with code, packages, tools and the type system. These principles are agnostic of the applied programming language, infrastructure and other technology in the tech stack. The following article in this series quite meticulously explains how we try to achieve our design goals.
State Transitions, Effects and Responses
Microservices are considered to own their own state. They may also share state, for performance or simplicity reasons. They may all be considered side-effects, but the article will consider State Transitions to be internal to the service, and hence inaccessible to other services except through the service boundary. We will refer to Effects as the state transitions external to the microservice. For example, changes to shared state, like a data lake, shared file storage, messages to APIs of an external system. Responses are the outgoing messages returned by the transport layer over the network, as a result of an incoming message.
Layers
Many architecture patterns have a theme of using a layered design: Onion architecture, Hexagonal Design, Ports & Adapters and Clean Architecture, to name a few. This section borrows and rephrases similar ideas. The reader is encouraged to research the aforementioned concepts for more elaborate analysis.
The seven layers discussed here are:
- Main
- Model
- App
- Contract
- Configuration
- Adapter
- I/O
Dependency direction
Each layer will depend on zero or more other layers. Circular dependencies between layers are not allowed. They are also not strictly required, even if they may be useful and applied a lot in some programming languages. As we’ll see, the dependencies constitute an acyclic graph, where the Contract layer plays a central role for enabling this through dependency inversion by means of decoupled interfaces. Most popular programming languages have some similar notion of interfaces that enables this.
The App and Adapter layers implement the interfaces in the Contract layer, such that all App and Adapter implementations remain independent of each other.
Each layer has zero or more independent instances, i.e. instances of the same layer may not depend on each other, except indirectly through dependency injection.
Main
The main layer provides the entrypoint of the executable program that will be run as a process on the containerised microservice.
Nothing depends on Main. Main depends on Adapter, Configuration, App and Transport.
Model
The model layer provides the domain-specific data structures that model the domain behaviour of the application. Closely reflecting the problem domain, it should be the purest core of the application. Adapters and Transport serve as translation boundaries to facilitate the Model’s agnostic position of technical complexities.
App
The App layer includes the domain models and the business logic required to calculate, derive and determine State Changes, Effects and Responses to messages, returned back to invocations of the Transport layer.
Contract
The Contract layer provides the interfaces necessary to decouple the App and Adapter layers from each other. Dependency inversion is achieved by this layer. As will become clear in the implementation below, this enables separation of concerns. It means other layers can be unit-tested independently, by mocking the interface of its dependencies.
Configuration
The Configuration layer is responsible for composing the settings required for various other parts of the service. Settings include data that should be set at time of deployment, and hence be static.
For mutable configuration that may be changed at runtime, be wary that microservices ideally remain stateless, for scalability. Rather, consider if this should be an internal state change, which should be part of the Adapter layer, which manages side-effects and brokers internal and external state changes. If configuration is considered to change as a result of a message from outside the microservice, it is handled by the Transport layer and further handled by the App and possibly its Adapter layer. It will then not change anything in the Configuration.
Adapter
The Adapter layer manages State Changes and Effects via I/O and other Adapters, invoked by the App layer, to facilitate decoupling and statelessness of the App. Adapter is the only recursive layer, because one Adapter instance may need another Adapter to fulfil its purpose.
Transport
A Transport is a special case of an Adapter. Hence, it’s excluded from the diagram, but nevertheless worthy of extra attention. Transport is single-direction broker for incoming messages, like requests, commands, events, remote procedure calls, queries, mutations, command-line calls, etc. It translates input to the language the App understands and invokes the App’s domain behaviour through its Contract. That is, it converts transport-specific input data structures to Models, such that the App can remain transport-agnostic.
I/O
I/O represents the external environment, that the program accesses via the operating system’s I/O primitives. The Adapters serve to encapsulate any side effects, and all such effects are effectively done by I/O.
Side notes
Third-party dependencies
Any layer can make use of third-party dependencies. An external library would still fulfil a role in a particular layer. For example, Main may invoke a package that loads in environment variables or command-line arguments to populate the Config data structure. These can be considered adapters.
Another example: Model may use data structures that are defined elsewhere, like a UUID package.
Logging
Logs are collected and can be considered Effects. But since logs are write-only from the microservice’s perspective, these are considered an exception to the rule that only Adapter instances may cause Effects.
A stricter view would be to consider the used logging system as part of the Adapter layer, and allow Transport, App, Main and any other Adapter access to this Adapter instance.
Transport response
Furthermore, a Transport Adapter can cause an Effect by means of its responses to incoming messages, which in turn can affect other system components’ behaviour.
Authorisation
Authorisation can be thought of as requiring its own layer. For this discussion, we presume this role is either fulfilled by Adapter, App, or both.
Whose role it is, can be determined as such: if multiple Transport instances invoke the same App’s domain behaviour, will they always require the same authorisation for that behaviour to be authorised? If so, the authorisation is part of the App’s domain behaviour. Otherwise, it apparently is specific to the Transport instance.
Further attention can be spent on the granularity of authorisation. Some domain behaviour may need to be authorised, i.e. for a State Change or Effect to occur as a result of a message received in the Transport layer.
Next up
In the next article, we implement the above design principles as they would apply in the Go programming language.
