Software Architecture and Agile. Are they both really compatible?
In this article, we will discuss how Software Architecture and Agile Methods can be used together to make the most of them.
What is Software Architecture?
There are tons of different definitions to describe what Software Architecture is. This one particularly appeals to me:
“Group of principles and constraints about how software solutions have to be built within a given boundary (usually an enterprise)”
Some examples of principles and constraints would be:
- Only programming languages based on the JVM
- Synchronous communication among components through REST APIs. Each API specification must always be written in Swagger.
- Asynchronous communication using the corporate message broker (i.e: RabbitMQ)
- Tracking (audit, logs) through ELK. Every message must have a specific format.
Ideally, this group of principles and restrictions will be influenced by the business that it supports. So it’s likely building software for a nuclear power station will differ greatly to building software for a marketing company.
Moreover, as with any software, quality attributes (non-functional requirements) must be included:
- High availability
The balance of these quality attributes should be set up in a way that the build solution demands. Building a small web application which will be used by just two users inside a LAN for query data clearly won’t be the same as a system that will manage clinical history (SECURITY), or software to control rail traffic (PERFORMANCE, AVAILABILITY, ROBUSTNESS). Quality attributes will obviously vary.
Therefore, architecting is applying these principles, functional and non-functional requirements to build a technical solution: components, interfaces, relationships, structure, and so on…
Martin Fowler describes Software Architecture as the decisions which are hard to change. A simple but brilliant quote.
Why Software Architecture?
As the number of software solutions grows, a degree of homogenization among these solutions is required. In most cases, this process has economic implications. Saving costs directly or indirectly.
Let’s go back to the previous part where we assumed that in an enterprise (or a subset in there) only JVM languages can be used. What leads a group of architects (or whoever) to make such a decision? Why do they restrict software development to a specific language? Furthermore, if we wanted our component communications to be done through REST APIs, the programming language won’t be a barrier because the service API itself will abstract the relationship to the language used to implement each component: high cohesion and low coupling.
The reasons, as we previously mentioned, used to be economical. What will happen if the whole platform would be built using seven different languages? Which skills a developer for such a platform should have? What will happen if we want to hire enterprise support for each language? How are these things run in production: monitoring, upgrading, infrastructure, and so on…? What will happen if we want to provide a corporate software framework to develop different components? Do we have to implement it in seven different languages? Those are just some of the problems that would arise without this constraint. Therefore, what could initially be considered as a limitation (using merely one programming language), becomes a way of saving costs in the future. Obviously, these sorts of constraints lead to drawbacks that have to be analyzed considering the enterprise’s culture.
So, Software Architecture will be worthy if it satisfies the architectural drivers (principles and constraints), providing the necessary foundations for the rest of the code, and will work as the platform for solving the underlying business problem.
What is Agile?
In essence, Agile is about moving fast and embracing change. It is continuous improvement. Agile is giving a good solution with a high degree of quality in a short period of time. It’s about removing waste (everything that is not worthy).
There are different agile methods like Scrum based around autonomous and self-organized teams. Scrum relies on an iterative (repeated cyclically, normally in short intervals) and incremental (gives new functionality) process.
But, how do these autonomous and fast-paced teams match with a group of principles and restrictions to build a software solution? Will the process of “architecting” a solution delay the continuous value delivery? Maybe the key point is, how much time do we have to spend on “architecting” in each iteration?
Definitely, there’s no easy and specific answer to this question. The most sensible answer would be “just enough”. The minimum amount of architecture to fulfill the principles, vision, constraints to build a solution. Once again, how much is “just enough”?
It depends on the project/product that is to be built. If we are in a very chaotic project with lots of changes probably the amount of upfront architecture would be lower than more stable projects where doing some design in advance would be easier. As a rule, it would be recommended to do some upfront design at the beginning of the project (Sprint Zero?) and before the first iteration. Besides, we should include an “Architecture review” as part of the Definition of Done (DoD) in each user story. Of course within the Sprint Planning, this fact has to be taken into account. It would be worthwhile if at least one of the team members (if not all of them) will be accountable to ensure both product development and Architecture are aligned.
If we just hope that the Software Architecture will simply emerge we could end up in a series of “refactoring sprints” because of the lack of upfront design, especially with large and complex projects. As Martin Fowler said, Architecture is a group of relevant decisions which are hard to change. Neglect with such decisions would lead to delays and huge technical debt, which is contradictory to Agile philosophy.
How much Software Architecture do we need?
Probably the answer may be something like, the minimum amount to fulfill the principles, constraints, functional and non-functional requirements as well as supply the basis to build a solution.
Let’s look at some different scenarios. If we were to build a solution to collect and querying data related to our customers’ clinical history, probably the Software Architecture will be strongly shaped by lots of politics about how to access data, obfuscation, certificates, tracking, protocols, etc…
On the other hand, if we were re-building a system because it’s unmaintainable and technologically obsolete, surely some principles about modularity, testability, new technology stack, etc… will appear.
Finally, lightweight Architecture will be needed when working on a new experimental product focused on a new market niche due to the uncertainty of the product itself.
Many enterprises have their own framework which implements some of the Architecture’s principles. Using it properly can be tremendously worthwhile when the agile team is building a solution. To do so, we should ensure that, at least one of the team members, will deeply understand such a framework ensuring efficiency, identifying flaws and helping to enhance it.
Software Architecture and Agile Methods in large enterprises
One may ask, how can autonomous and self-organized teams make the best of, as well as taking advantage of the Software Architecture? How can we let Software Architecture evolve without holding up the teams?
Autonomous and self-organized teams use to be focused on a specific area within the whole enterprise, which is great. Nevertheless, it’s quite difficult that such a team will have a global insight that let them anticipate problems that might appear outside its context.
Likely, they won’t understand the whole system and this fact may lead them to build redundant solutions or heterogeneous designs within the enterprise’s global context: maybe other teams had dealt with the same problem before. That’s what makes it desirable to have one person or a group accountable to look after the Software Architecture at a high level.
On the other hand, improve and optimize the reference Software Architecture, and the innovation process itself shouldn’t be a centralized task owned by a group of people but something that could emerge from any team within the whole organization. Obviously, every new idea (capacity, innovation, component, etc…) that will emerge have to be managed and included in the reference Architecture in a centralized way.
There is an interesting concept within the SAFe Framework that tries to take advantage of both approaches called Agile Architecture. The starting point is a bit of Software Architecture (upfront design) which is retro-feeding with the emergent design of the autonomous teams.
Doing so we reach two benefits:
- Having a reference Architecture which helps us to build our solutions
- Let the teams have a degree of innovation that, at the same time, will feed the Architecture and will allow other teams to take advantage of that.
How can we keep Software Architecture and Agile Teams to be aligned?
When we mean agile and autonomous teams we also refer to multi-skilled teams. Such teams are composed by dev-ops, scrum master, product owner, frontend developer, backend developer, QA, technical leader and so on. Here is where the technical leader plays a critical role.
The Technical Leader or Software Architect will be accountable to share the Architecture’s Vision and Roadmap within the team. He or she will be in charge of solving technical issues, handling the innovation requirements and synchronise them with the Reference Architecture.
Of course, the Software Architect isn’t an authority but a mentor who teaches and helps to improve the effectiveness of the Agile Team working quite closely with them. Moreover, he or she must have a deep understanding of the Reference Architecture and has to keep in contact with the rest of Software Architects whose work with the other teams (Software Architecture Chapter?).
If we want to have autonomous teams that won’t result in chaos, communication and alignment of principles will play a key role. Coherence and consistency can only be achieved by introducing a degree of control.
- Book: Software Architecture in Practice (Sei Series in Software Engineering)
- Book: Software Architecture for Developers
- Book: Scaled Agile Framework (SAFe)
Agile methods and Software Architecture aren’t incompatible but supplementary. Such Architecture shouldn’t be considered as a limiting factor over the team’s capabilities but as an enhancer within the Agile Supply Chain.
Many thanks to Martin W. for helping me with this article :)