Emergent Architecture: Architecture in the Age of Agile
Throughout the world, pre-Agile software development was commonly done using a waterfall-based delivery model made up of a sequence of distinct activities performed by specialists. Hand-offs between stages are usually weighty documents — the keys required to unlock the next stage. All these models were variants on a common theme:
- High-level design
- Low-level design
Getting from start to finish would be a matter of months or maybe even years.
The key responsibility of the Solution Architect in this world is to produce the High-level design — an artefact that ostensibly tells the dev team enough about the structure of the solution that they can take forward, but also reassures multiple external stakeholders that the solution is fit for purpose. Those external stakeholders are the Corporate Guardians — people with accountability for distinct concerns of the Enterprise which also includes the Enterprise Architects who own the Technology Strategy.
The distinction of responsibilities in the lifecycle meant that Architects didn’t need to be physically located with the dev teams — an arrangement that has often led to Architects being too abstract or academic to consider developer concerns resulting in accusations of Architects handing out proclamations from their “Ivory Towers”.
Agile the Disruptor
Then along comes Agile with its fancy ideas! Rather than taking several months to deliver something we thought was the right solution at the beginning, let’s go for smaller batches of change. With smaller batches of change, we can iterate towards the goal and realise value earlier than with Waterfall. We embrace that the goal we eventually hit may not be the goal with which we started the journey. We have the power to change our minds and plot a new course. I’m sold, and there’s no going back for me.
The key to making smaller batch size work is to minimise the hand-offs necessary to deliver the batch. How can Architecture, with its role of keeping the Corporate Guardians pacified, support this without becoming a bottleneck? The answers are guide, trust and support.
- Guide — to clearly communicate solution options, along with risks and strategic alignment. This needs to be done with the right level of detail — not so high that it’s meaningless, and not so low that it’s over-detailed
- Trust — Empower delivery teams to make appropriate decisions
- Support — to be there for the delivery teams, to provide additional guidance where required, to reassure or review where conflict emerges, and to hear and evaluate ideas and challenges from the teams
Emergent Architecture = Intentional Architecture + Emergent Design
Intentional Architecture is the declared, desired target state. Note I say “target”, not “end” because there rarely is an end for software — a point at which no further development is required. The target state can and will evolve, but it’s always a projection of where we now believe we want to be at some point in the future.
With Agile and its small batches, delivery teams may make decisions that diverge from the Intentional Architecture. And that’s fine when done openly, with honest debate. Architects need to be aware of Emergent Design as it has potential impacts:
- Embrace — the Emergent Design is a valid transition state of the Intentional Architecture
- Eliminate — the Emergent Design meets the needs of today but introduces new technical debt that needs to be paid back to reach the Intentional Architecture (rework required at some point)
- Rethink — the Emergent Design introduces an opportunity to do things better. The Intentional Architecture needs to be updated.
A way I like to look at it is that Emergent Design represents a fast-feedback loop for Architecture, and thus the Architecture itself becomes Emergent.
The Agile Architect
The Agile Architect is one who accepts Emergent Design as a necessary and beneficial part of the process. Trust is a two-way street — teams need to be able to trust their Architects and Architects need to be able to trust their teams. That trust comes from familiarity, from being in the game, not outside it. The Agile Architect must join the devs in the trenches, providing valuable support and absorbing feedback.
Finding the right level of guidance again comes from familiarity. Embracing Agile fast feedback loops. If a design doesn’t land right the Agile Architect has to adapt their approach and try again.
This means a successful Agile Architect has to be an effective communicator, able to adapt to the audience, to be understood and just as important to be able to understand.
The days of Architects commanding from their Ivory Towers are long gone…