Enemy of the Target State: How Agile Product Development Redefines the Meaning of Target Architecture
Back in the age of Waterfall, architects would normally define their target architectures after having reviewed a list of functional and non-functional requirements for the full product to be built. Architects would argue that a solution needed a problem, and if the requirements were unknown or not known enough, no target architecture could be defined.
That argument was not flawed in any way. In fact, that a solution needs a problem is logic that still applies today and that will always apply.
But what does that mean to target architectures now that we manage and develop products in Agile? For a newly conceived product about to be implemented, for example, we have a vision for the product and normally an explanation of its first version as a minimum viable product (MVP). Whatever form that vision and MVP description take, those are our requirements. We should not expect at that stage, however, detailed requirements for a target product beyond the MVP. And after the MVP, the same applies to the next release of the product.
The requirements that we can count on define the problem to be solved up until the next production release of the product. True, a product would normally have a backlog of features to be implemented subsequently, but those are not always guaranteed to be in scope as it all depends on that feedback loop with the customers based on their experience with the product. Remember, the team consciously chooses not to define everything upfront about what their product will or will not do.
This means there is not really a fully predefined target product beyond the MVP or whatever the next release of that product is. There is no fully predefined target problem to solve. It follows that there cannot possibly be a fully predefined target architecture either.
Architects have a natural and understandable reaction to those conditions. That is to conclude that the role of a target architecture of course still exists, and that it “just” needs to be defined at the beginning of implementation and then updated as needed ahead of every product iteration.
There is a lot of true in that. However, an architecture conceived and maintained merely that way is at the same time unnecessarily short sighted. That is the main issue that I’m going to addressed in this article.
Good architecture is architecture that is easy to change. By not accounting for the likely future evolution of the product, the architect fails to introduce in the architecture the right abstractions, points of extension and other characteristics that will make it as easy as possible for it to change and evolve with the product.
Accounting for the likely future evolution of the product is easier said than done.
Future proving is positive when it makes it easier for the evolution of the architecture to follow that of the product. Future guessing or premature anticipation of likely requirements is negative, as it often leads to over-engineering which is a fatal flaw in Agile and Waterfall alike.
For a reasonable determination of the likely future direction of a product, there is in fact a powerful place to start. That is the product vision. While the specific features are meant to be defined and prioritized incrementally and not from inception, the product vision must exist from the outset. If it is not being articulated to you as the architect in the team, it has to be. Ask the product manager. What is it that makes that product special, different or otherwise superior to its competitors? Who is the product for? What value is it meant to deliver to its customers and its business? Without that clarity, the product would be unlikely to have a business model to justify the investment of being built in the first place.
It is important for you to observe that the MVP or any of the individual product releases are not your primary concern. In fact, for the product manager those releases should not be their primary concern, either. For both of you, it is the actually the vision that you should truly focus your energies on at this stage. The product manager focuses on the product vision, and you focus on the architecture vision.
Product and architecture visions go hand in hand, and together determine the direction that the product will follow from a customer value and technology enablement stand points.
To be clear, it is not that the we should not be thinking of the MVP — that is not at all the case. Spending enough time to define the product and its architecture for the MVP, as well as subsequent releases as we get to them, is of critical importance, too. However, it is only with a clear vision of where you are heading to that the team can make the most informed design decisions in each product iteration.
To that end, as the architect you should express your vision through the articulation of a north star architecture. That is used as a guide for the wider team in the definition of the architecture for each one of the product releases. How do we know that the architecture underpinning each release is evolving in a way that helps the team enable the ultimate vision for the product? That is because they are moving in the direction of the north star architecture, which was in turn defined based on the product vision.
In case you are wondering, north star architectures are not the same as the “final” target architecture in the Waterfall sense. That, as explained earlier, would imply a known final target product, and a final known problem to solve, which do not exist in Agile product development. The north star architecture is rather the general direction to be followed by the architectures of each release, so that they best support the progression towards the product vision.
The following questions illustrate concerns that you address with the team for at the initial stage of development. Notice how their answers allow you to derive important characteristics for the north star architecture, without falling into the trap of attempting to define upfront a set-in-stone target state architecture for the product. Additionally, notice the concerns that are intentionally deferred to be addressed in the architectures underpinning each one of the product iterations, and not in the north star architecture.
1: MVP or any specific product release aside, what are the high level feature sets or business capabilities that we expect to enable at some point in our product roadmap as per the product vision?
Understanding the expected high level product feature sets (or, business capabilities for enterprise products, to use common organisational speak) allows you and the team to derive the list of services in the north star architecture and where to draw the line between their respective responsibilities in the overall solution. You do not normally define in the north star architecture the specific operations within those services or any specific interactions between them as you would have done for the target architecture in Waterfall. You do not have, and do not want to have, enough upfront requirements to do that anyway. But you do define the separation of concerns and the architectural approach to avoid unnecessary coupling between the capabilities in their implementation. Note that the the MVP and future product iterations will certainly require the detailed service operations and service interactions that you are not including in the north star architecture. Those must come in the articulation of the architecture for those iterations when you and the wider team get to them, and not before then.
2: What assumptions can be made on how the product fits in the existing architecture of the organisation, and how that architecture is likely to change over time?
Services derived from the high level feature sets, or business capabilities, as described above often require integration with preexisting services and systems of the organisation’s architecture as their dependencies.
In your north star architecture, do account for those and, additionally, determine where to insert protections against those dependencies changing in any way. System interfaces may change their exposed data model, the integration method itself may change, or even the entire system itself may be the subject of replacement. That protection normally takes the form of layers of abstraction, such as adapter services or events.
But do not waste any time defining in the north star architecture the detailed interactions of a service in your solution, or its adapters, with any of its dependencies. Those may very well change ahead of one of your product iterations actually requiring it. That is after all why you introduced that layer of abstraction in the north star architecture in the first place. So, instead, defer that task to the product iteration that actually requires it.
3: Based on the product vision, what assumptions on the non-functional scope of the product are likely to change relative to the MVP and as we progress through product iterations? For example, it may be that an MVP is targeting customers in a particular country, in which case the question is whether it is expected to have customers in multiple countries at some point.
If the answer to the sample question is that multiple countries are expected, then you and the team reflect on what the implications are for deployment, data storage and other factors. Most importantly, you reflect on how the the architecture should not preclude a future evolution from catering for those, and then apply the desired characteristics for that in the north star architecture. For example, if you and the team conclude that multi-region deployment is likely to be of value in future iterations, the north star architecture describes it as such. The team might then want to implement, for example, adequate region parameterisation in their pipelines, but the architecture underpinning the various product iterations does not support multi-region deployment until it is actually required.
There are certainly many additional questions for you as the architect to consider with the team. Besides, as you would expect, those questions also vary depending on the nature of the product that your team is going to implement. The point here however is that there will always be such questions to help you inform the north star architecture, and not asking them or not considering the implications of the answers would make your architectures unnecessarily short sighted and harder to change.
So, do not fear prosecution, do not seek exile. Be the enemy of the state! The target architectural state, of course. Drop the idea that there is a single target state that is set in stone since the inception of the product, or even only two or three interim states. There may potentially be many architecture states throughout the life of a product developed in Agile. But do set and promote an architectural north star to guide the decisions in all of them. In every product iteration, work with the team on how to move the architecture as per the north star direction. We may consider the north star architecture a redefined meaning of target architecture, one that gives us the direction and also the flexibility to adapt of Agile, but in any case you must embrace it. It is critical for the success of the team and the team’s product.
