Agile Architecture — Make decisions, make mistakes. Just don’t lock yourself up

In the “do, learn, repeat” cycles of Agile product development, analysis-paralysis is not acceptable, but mistakes generally are, if you can recover from them. What does that mean to architecture decision making? And what are the cards to get the product and its architecture out of jail when hypotheses and decisions turn out to be wrong in the real world?

note: author is an architect, not an actor (clearly)

“We cannot afford to be wrong!” Wrong. Those in Agile product development teams would rarely say something like that. Try to make sure that you are definitely right every time that you are about do something, and good luck launching any product, or any feature even, anytime soon.

No amount of upfront research and analysis can absolutely guarantee the success of a product or a feature with their customers. As a product feature is considered for implementation, an Agile product development team puts just enough thought into feasibility, value and likelihood of success. Instead of delaying implementation until there is somehow a sense of absolute certainty in any of those, they allow for the possibility of mistakes, and know how to learn from it to improve future product iterations.

In Agile, doing and learning is more important than absolute certainty, even if the latter were at all possible. That mindset applies to everyone in the team, and to everything that they do. The architect must feel comfortable that the architecture decisions that they make with the team will not always be proven to be the best ones, or even good ones, in the real world. That is generally okay, especially if the architecture can iterate towards better decisions with as little disruption to the product and to the team as possible.

That mistakes are acceptable, analysis paralysis is not, and that this does apply to architecture decisions is my main point in this article. Beyond advocating for that mindset, however, I do want to be more concrete about it. For that purpose, the following list is a set of principles that I derived from practical experience — my own and my teams’ — that illustrate how that mindset manifests itself in practice in the day to day activities of an architect in his or her role in the product team. The principles span decision making time and the recovery from decisions when they turn out to be “not quite right” or even just bad in the real world.

1. Don’t Overthink: as an architect, when you consider solution options A and B, do think them through, but don’t overthink the decision. If options do look similar in their pros and cons, it is likely that they are indeed going to be similar in their effect on the product. Be confident, pick one with the team and see how it goes. After all, there is a point where it is only implementation and the real world that can give you further clarity and prove you right or wrong, not paper based exercises. Besides, exaggeration of paper based exercises would likely impact pace and disrupt the team’s ability to meet their goals for the sprint for not enough in return, and that would definitely be wrong.
2. Don’t Over-Anticipate: it is often the case that the decision to be made turns out to not really be between options A and B, but on whether to go to A as a stepping stone to B, or straight to B. The typical scenario is the following. Option B is perfectly aligned with the north star architecture and delivers all the technical components that enable the team to meet the ultimate vision for their product. Option A, however, is a step towards that same north star that enables the features of the current product iteration, but which is otherwise incomplete relative to the full technical component set underpinning the ultimate vision. The decision here should be straightforward, and not justify lengthy deliberation. Option A wins because it represents the architecture iterating in tandem with the product towards the vision. In contrast, option B would mean premature engineering at best, and over-engineering at worst, since the vision can change as part of the short feedback loops of Agile product development.
3. Prove it, if you really have to: it is of course possible that the team doesn’t feel confident proceeding with a decision without experimentation within the team through proof-of-concepts (POC) or technical spikes. Sometimes that does turn out to be the shortest learning path possible, as you can prove something to be right or wrong within the team and without taking it all the way to customers in production. So, by all means, run a POC when it make sense. But make sure there is an actual concept that is important to prove and this not just you and the team in fear of being wrong without a solid reason.
4. Don’t Overcommit: if there is a decision involving the selection of a 3rd- party component, such as a SaaS service, for example, do make an informed enough decision, but allow yourself and the team to be wrong or change your minds afterwards about that service. Don’t prematurely lock yourselves in long contracts or on pricing models based on usage demand that your product is not even guaranteed to have. Favour, for example, consumption based models like those based on the number of API calls — at least, until your team’s product matures and you become more certain that the service was the right choice.
5. Create the Easy Way Out: when integrating that 3rd party service, or any service for that matter, you should consider the exit route for your reliance on it. If real world experience later on indicates that you are better off with a different service, you and the team will want any re-work to be kept to a minimum. In fact, there’s always an exit route: you can replace the service, of course. But with the right abstraction layers, like your own APIs or events that hide the actual implementation contracts that exist behind the scenes, you can create a way out that is as short and easy as possible.
6. Write it Down: “working software over comprehensive documentation” as per the Agile Manifesto does makes sense to me. But do write down the rationale for your architecture decisions. If later on you and the team find a reason to revisit them, understanding your thought processes on the original decision will certainly help you to re-assess the decision more quickly. You may even find that the assumption and conditions that you considered in the original decision no longer apply.

It is possible for an architecture derived from well informed decisions to not pass the tests of the real world. How big a problem that is depends of course on the severity of the real world tests that it failed. But in any case, as the small (and certainly not exhaustive) list above illustrates, overthinking decisions and over-anticipating solutions is often not justified. There are normally approaches to be taken, and characteristics to be introduced in the architecture that can make it easier for it to be adapted if it does turn out to be wrong. Those can be the “get of jail free cards” for the architecture for your team’s product, or as close to free as they can possibly be.

Make good architecture decisions. Make inevitably some mistakes, too. We can generally afford to be wrong. In Agile product development, not making a decision is the mistake that you cannot recover from.