From Prescription to Direction, The role of the Architect in an agile organisation

Enterprise, Software, Systems — architects in the software industry come in various shapes, sizes and titles but in essence their role is the same: to allow the organisation to keep moving quickly as their software systems grow and become more complex. It’s easy to sprint once, especially in the earlier ‘green field’ days of a project — but to do this repeatedly requires careful planning, research and vision.

In the short history of our industry, we tried to achieve this by overplanning and slamming all sorts of bureaucracy and process at the problem — Waterfall anyone? This was also reflected in the way we structured our teams, with technical leadership coming from the top down.

In the agile world, a lot of this has changed, as detailed preplanning has given way to iterative design, top-down management has evolved into empowered, cross-functional teams. As the pieces fall, where does the role of the architect lie?

Shorten Lead time

One of the key changes in making architecture more agile is to work in the same way we do in development — iterative improvement. The first focus should be on the macroarchitecture views. Once we have identified the larger building blocks of the system, we should be able to (at a high level) come up with research tasks to identify which parts can be bought, vs which parts the team will need to build.

Collaborative Research

At this point, architects would typically retreat into their ivory tower and conduct detailed comparative analysis and proof-of-concepts. This is a very healthy exercise that goes a long way into selecting the best options, however it is also a very time consuming one that risks sending architects down the rabbit hole in search of perfection. At this point, I think there are two ways of mitigating the timing issues that this may cause. The first one is to leverage members of the development team in the research process. Utilising pair programming while setting clear desired outcomes which are tied to the project, the architect can partner with members of the development team to conduct the research and build the POCs. This serves two different purposes other than saving time — it will give valuable context to the team, and it also increases ownership of the chosen solution since the team was part of the selection process.

Ongoing Research

The second way we can mitigate the time cost of research is to do it preemptively. Attending innovator/early-adopter conferences, such as QCon, provides a good outlook into the way the software landscape is evolving. The architecture team can then begin looking into key upcoming technologies/products identified which will increase team knowledge but also reduce the time needed should any of these suddenly become necessary.

Focus on the Macro and Iterate

Once the tentative macroarchitecture has been defined, it’s important to drill down into some areas of the product. Yet, it’s necessary to realise at this stage that drilling down into every part is impossible since this would require more time than the process typically allows for. Falling into this trap will create a bottleneck for the team.

Therefore, we first need to look into the components that will be developed next by the team — and I personally believe that certain main areas should be prioritized, such as integrations (APIs in particular) and areas of complex business interaction. We can also make our life easier by using tooling that integrates specification with actual development — for example, Swagger/ RAML API Documentation. This will ensure that we can hook code/qa with the specification so there is no drift. Code generation tooling can also be used for kick starting development, as well as to keep it in sync.

Enable Craftsmanship

Managing complexity is not only a documentation exercise, as mindset and culture play a big part too. Empowering teams is key in increasing ownership — which tends to be treated with unicorn-like respect. However, throwing responsibility at a team without also providing it with technical support and the ability to fail can have serious ramifications. Fear of failure, eagerness to please, lack of direction and pressure from management can put us in a position where complexity grows due to the individualistic (and possibly overengineered “just in case”) development. As architects, it is our role, now more than ever, not just to empower the team but to guide and take certain key decisions to steer it and the organization in the right direction. Architectural decision taking can be a delicate balancing act, since we need to be mindful of team empowerment while also providing high level direction. Done properly, this could be a good learning experience for developers who they view as a positive safety net, rather than an oppressive dictatorship.

Guidelines vs Specification

In architecture, there is the potential to create a never ending stream of highly detailed specification that describes all aspects of the system being built. Despite being desirable for certain scenarios, for many systems this serves to lengthen the time needed before coding can start. It also serves to create a “not my problem” mental fence, where developers may switch off their own mind and rely blindly on the specification. This goes directly against the idea of empowering the team and reducing the time needed for architecture. We can solve this by producing specification selectively, and relying on “guidelines” or macroarchitectural rules/ standards for others. This will require developers to fill in the blanks and, in doing so, provide a controlled environment for development.

Keeper of the flame

Lastly — we need to focus on people. It’s very easy for any software professional to focus on technology, the tools we use and the Things™ we build. However, the people who form the teams that build these Things™ are alive, evolve and grow over time. As the team develops, cultural and tribal knowledge are acquired and built into the way it works and the Things™ they build. As architects, we are usually in a position to mould this culture and promote it positively — with our job looking to ensure that this is not lost over time. This allows us to guarantee continuity as people, tooling and technology change.