Evolutionary Sociotechnical Architecture Patterns

We say that architecture should be evolutionary — continuously adjusting itself to better serve business needs. What does an architectural evolution actually look like, though? And how are we supposed to know when and how to evolve our architecture?

Even more importantly, is it possible to evolve an architecture once we have many teams who all own a different part of the system? How easy is it to evolve an architectural boundary that crosses many teams?

Patterns help us to answer all of these questions. Patterns teach us to recognise signs that indicate a need for evolution, and patterns give us a set of options with associated trade-offs for how to evolve our architecture.

In the last post, I introduced structural sociotechnical architecture patterns, emphasising that economical, political, technical, and social factors that must all be considered when making design choices.

In this post, I’ll introduce a few evolutionary sociotechnical architecture patterns, where every design decision must again consider those fundamental factors.

Evolutionary Pattern: Slice and Scale

Pretty much all products, services, and capabilities transition through an evolutionary lifecycle. They start off as novel new ideas with opportunities for differentiation, then they become table stakes and standardised, and then they fade away*.

As products evolve through their lifecycle, so too should the software architectures and the organisations that enable them.

The Slice and Scale evolutionary pattern can be applied when an existing sub-capability gains significant momentum, for example: sudden growth, high customer demand, or strategic significance.

Slicing out the sub-capability into it’s own context allows the sub-capability to be scaled independently i.e. more effort can be directly invested into the capability without having that investment diluted by needing to support related capabilities.

Remember, a sociotechnical context is a business capability, the team responsible for the capability, and the technical systems needed to deliver the capability.

As an example of slice and scale, consider a music streaming platform business. In the early days, they have a music discovery team whose responsibilities include catalogue, search, and recommendations.

All three of those capabilities are owned by a single team, because, economically, the business does not wish to invest in a full time team for each of those capabilities?

In those early days of the business, differentiation was about having a large collection of music and a stable, scalable technical platform. Those were the areas where an advantage could be gained in the market.

As the competitive landscape evolved, however, a rich catalogue and a scalable streaming platform were table stakes. The differentiator had become advanced discovery — helping customers to find the content they would enjoy the most.

Now, search and recommendations are the differentiators and the business needs to invest heavily in them. Slicing them out as separate contexts, each owned by a team who can devote their full attention, is aligned to the business strategy of needing to out-innovate their competitors’ discovery capabilities.

Trade-offs

Each of the capabilities becomes a largely decoupled value stream allowing the business to make more granular choices about how it can invest resources in each capability.

However, achieving the decoupled state is likely to involve resistance.

Technologically, the system may be too tightly coupled for a smooth separation. Equally, breaking up a team who are performing well together may cause unrest. Output will also be reduced in the short-term. Never an easy sell.

Up-front analysis, design, and care play a huge part in minimising these pains. If a team can anticipate how the market or business strategy may evolve, it may be able to pre-empt future changes.

Also, these challenges are a potent reminder that investing in a team who produce well-designed and maintainable code gives your business immense strategic flexibility.

Evolutionary Pattern: Slice and Scatter

Scaling a business capability is not the only evolutionary pressure. Another evolutionary pressure is the need to improve team productivity by reducing cross-team collaborative overheads.

One pattern that realises these benefits is the slice and scatter pattern. By slicing up a capability that has become a bottleneck, the bottleneck is removed and each team owns the parts it frequently needs to change.

One example I’ve seen in a variety of domains, from government to travel, is having a dedicated rules context. Often, this will be implemented using some type of generic rules engine or BPM system, but not always.

The result is that that each business capability is largely anaemic and calls off to the rules service to check business rules. Consequently, the rules team is the bottleneck who have many teams blocked waiting for them to make changes. This is a political battleground.

In every case I have encountered, there has been no need for a dedicated rules context. It has always been possible to slice up the rules context into smaller sub-components and move those sub-components into the contexts that rely on them.

Trade-offs

While this pattern can reduce the unnecessary overheads of excessive cross-team collaboration, the journey may not be smooth or pleasant.

When the bottleneck context is split up and scattered, there will be one fewer team. In a world where managers are focusing on increasing their head count in order to climb the org chart, expect a lot of resistance.

In addition, there is again the problem of breaking up a team who may be productive and happy working together. Think also about the case where a team inherits some low-quality code from another team and is expected to support it.

In organisations with strict planning and rigid org charts, this evolution may be impossible in some situations. Technically, it may also be unfeasible if the technology systems are disparate and the merge phase requires unjustifiable effort.

As an example, one organisation I worked with had a rules context with thousands of rules hard-coded in C++. Porting that to other languages existing teams were developing in was not a quick fix before lunch.

Evolutionary Pattern: Slice and Merge

Sometimes, the bottleneck can be removed by extracting the sub-capabilities of tightly-coupled contexts that co-change and merging those sub-capabilities into a brand new context.

Trade-offs

Economically, the benefit is again improved efficiency by reducing the amount of coordination between teams.

Equally, the challenges are similar with regard to reshaping team boundaries and affecting reporting structures. In this case a new team is being created, so there are opportunities for people to take on increased responsibility rather than lose it.

Sometimes, the new context will be too small to justify it’s own team, so it will be allocated to an existing team. As we know, nobody wants to take on other people’s messy code.

There is also another hidden danger with this pattern. For a piece of work that runs end-to-end, instead of two teams needing to collaborate, now there are three.

As always, sociotechnical architecture is a high-stakes game where one thoughtless move can cause irreparable damage.

* To learn about product evolution in more detail, check out the works of Simon Wardley and Barry O’Reilly,