Evolutionary Pressures Forcing Software Architects to Adapt
We are currently witnessing a generational shift in the evolution of digital product development. Software architects who wish to thrive in the forthcoming era must be prepared for significant refocusing of their priorities.
Evolutionary pressures of the current generational shift are driving software architects to become customer and product-centric organisation designers. Those who do not adapt will cease to be effective or will stifle their organisation’s ability to develop digital products and services, or quite likely both.
However, current evolutionary pressures also amplify the need for architects. We are now essential players in creating joined up, highly-aligned sociotechnical systems positioned to harvest the customer feedback goldmine. Those of us who adapt will help our organisations prosper.
In this post, I’ll briefly talk about the three most significant evolutionary pressures presenting existential challenges — and substantial opportunities — to modern architects: customer centricity, infrastructure commoditisation, and connected experiences.
I’ll then introduce some of the most popular organisation design and strategic practices used by architects who have adapted in ahead-of-the-curve organisations and are flourishing.
If you’d like to learn more: check out my new book Designing Autonomous Teams and Services, or come to my talk and meet me at O’Reilly Software Architecture Conference in London.
Digital Evolutionary Pressures
Evolutionary psychologists use the term evolutionary mismatch to describe adaptations that provided an advantage in one environment (the environment of evolutionary adaptedness [EEA]), but no longer confer an advantage in the current environment — possibly even conferring a disadvantage.
When food was a scarce resource, it made sense for hunter gatherers to eat at every opportunity, not knowing when the next chance would arise. But in the modern western environment where food is abundant, eating disorders resulting from the adaptation to eat at every opportunity cause serious harm.
As software architects, we need to be aware of the changes to our environment that may lead to evolutionary mismatches. The unprecedented rate of change in digital product development causes evolutionary cycles to occur far more rapidly than we realise.
Customer Centricity
I tire of hearing the cliched boasting about how companies put their customers first and exist only to make their customers happy. However, do not confuse the false, marketing-hype with true customer centricity.
The need for customer centricity is a simple economic formula; research and anecdotes continue to unanimously point towards customers as the best source of new product and feature ideas. The more you involve customers in your product development process, the better your products will be.
So why has customer centricity suddenly become an evolutionary pressure?
Firstly, the field of product management has developed at an incredible rate over the past few years. Successful organisations began adopting a product culture, and the rest of the industry followed out of necessity to stay relevant.
A new age of customer centric product managers were inspired by movements like the Lean Startup that focus on maximising the speed of learning by involving the customer.
Software engineers invented practices like continuous delivery to increase the speed of delivering value. The new age of product managers brought a product focus to the process — now known as continuous discovery and delivery or dual-track agile.
As architects, our goal must be to design sociotechnical systems optimised for continuously discovering and delivering value to users.
Infrastructure Commoditisation
Many of the advancements in product management that led to the evolutionary pressure of customer centricity were facilitated by another evolutionary pressure — infrastructure commoditisation.
The barrier to entry for creating digital products was continually lowered by the emergence of AWS and the cloud. Previously, organisations needed to build and manage their own infrastructure. The expensive up front costs were a deterrent to many aspiring entrepreneurs who lacked capital.
With the emergence of the cloud, expensive up front costs were eliminated. The reduction in costs and risks of building digital applications increased competition in the marketplace — hence the rise of customer centricity.
However, architects must also be aware of another impact of the commoditisation of compute — infrastructure is no longer an opportunity for game-changing competitive advantage or cost reduction.
Prior to cloud, tech companies with advanced infrastructure engineering capabilities could deploy software more frequently (enabling faster product development) and for reduced operating costs. Now, these best practices have been built into the services offered by cloud providers.
Over time, cloud services will swallow even more infrastructure concerns that organisations typically had to handle themselves. As a result, organisations that do not move to the cloud and focus there efforts on areas of differentiation will be at a disadvantage to those who do.
As architects, we must be aware that the opportunity to differentiate from competitors and gain competitive advantage is at the top of the chain — advanced software and product development, not the commoditised bottom of the chain — managing our own infrastructure.
Connected Experiences
We now live in a service design world where we expect not just applications on our smartphones, but a seamless user experience connected across all of our devices.
When we send an email via the web UI, we expect that email to appear on the smartphone app instantaneously. When we wear wearables like the fitbit, we expect the device to transmit information that we can inspect on our laptops, and tablets.
Services that do not provide a pleasurable experience across multiple devices will be replaced by those that do. As architects, we must ensure we are optimising for our organisation’s ability to design and deliver connected experiences
Delivering connected experiences has profound implications on how organisations should be architected. How can many different teams (tens or hundreds), each building different parts of a service, collaborate to provide a consistent user experience across many devices?
There are a variety of approaches for designing teams and software architectures — each trading user experience and speed of innovation in certain areas of the system.
For example, creating dedicated frontend teams ensures a single team owns the user experience for a specific device (e.g. Web, Android, iOS).
Consequently, though, any changes that require front and backend changes will require the collaboration of multiple teams who may have conflicting organisational priorities or varying availability to complete the work — leading to one team being blocked by another, and critical work items stuck in the system somewhere.
If work frequently spans front and backend teams, the costs of collaboration will be high and innovation speed will be slow. Bureaucracy is certain to creep in.
As architects, one of the most vital roles we must play, is not just designing connected software systems, but coaching our organisations to find team and technical boundaries that find the optimal trade-off between user experience and innovation speed.
How Architects can Adapt and Flourish
I am excited by the evolutionary pressures we are facing, and I see no reason why architects should fear them.
The new era we are heading into places a premium on building engaging products that solve real problems. We are focusing more on satisfying real end user needs and less on managing accidental complexity.
Here is a brief introduction to some of the popular theories and practices used by architects who are flourishing in the current environment.
Aligned Autonomy
To be user centric and innovative, we need to complete innovation cycles — gaining customer insights, implementing them in software, delivering the solution, and validating the solution — as quickly as possible.
We need to design teams with the ability to own all or as much of the innovation cycle as possible so that every decision is made in the shortest amount of time. We need autonomous teams aligned with specific capabilities or use cases.
In a connected experiences, service design world, we know that a seamless user experience is mandatory. So how is it possible to have the entire user experience owned by one autonomous team?
Trick question — it’s not.
To be effective as architects, we must therefore create aligned autonomy. Not only do we architect the boundaries of teams and software services, we also must design links between them — designing the flow of information between teams so they can make good decisions locally with full situational awareness of what other teams are working on.
The approaches I encourage for enabling aligned autonomy include cascading visions/objectives (where cascade is used loosely to mean influences not dictate), cross-functional workshops, regular show and tells, and cross-team pairing (see Enabling Coevolution of Organisational and Technical Boundaries for further discussion).
Theory of Constraints
When innovation speed is the dream, dependencies are your worst nightmare. Coordinating multiple teams, managing the flow of work through them, and integrating the systems they build is a high stakes game of chess. One wrong move can introduce dependencies that bring programs of work to a standstill.
Teams with different backlogs and organisational priorities become embroiled in endless meetings and bureaucracy, spending most of their time trying to understand what they should work on, rather than actually delivering value.
As architects, we should be focused on guiding the design of organisational and technical boundaries that strategically minimise the cost of dependencies where we need the highest innovation speeds. Theory of Constraints (ToC) helps us solve this problem.
ToC teaches us that bottlenecks are the limiting factor in the performance of a system. If we coach our organisations to find the biggest bottlenecks and eliminate them, we will foster continuous improvement of the performance of the entire system.
ToC gives us the mindset to look for bottlenecks in all kinds of systems — both social and technical.
Thinking in bottlenecks will transform your perspective. You will be shocked at your new way of seeing the world. I highly recommend you read The Goal or the new graphic novel version of The Goal.
Strategic Domain-Driven Design
At the heart of Domain-Driven Design (DDD) is the principle of focusing on patterns in language to identify boundaries in systems. As a result, used properly, DDD is a powerful tool for finding boundaries in problem domains.
By aligning our teams and software services with the boundaries we identify in the domain, our teams and software services will have fewer dependencies and greater autonomy.
DDD encourages cross-functional collaboration to build a shared, evolving model of the problem domain. The model includes a shared language and boundaries — known as bounded contexts.
When you talk about the domain, when you draw diagrams, when you design teams, or when you architect software systems, you use the same model (the same language and boundaries).
As architects, we should play a key role in facilitating the design and evolution of the conceptual model. We should encourage use of the shared language, and organise workshops to regularly review and refine the model.
Two of the most effective DDD practices are domain storytelling and event storming. Both of these activities provide workshops formats for creating shared conceptual models, building a common language, and identifying boundaries in the domain.
You’re Not a Software Architect Anymore…
To remain effective, you must learn to co-design and coevolve organisational and technical boundaries. You’re not a software architect anymore, you are a sociotechnical systems architect.
If you’d like to learn more about any of the ideas in this post, check out my latest book, watch some of my past talks, or follow me on twitter. I’ll also be speaking at the following conferences and would love to meet you in person.
- Swanseacon (September)
- O’Reilly Software Architecture, London (October)
- KanDDDinsky, Berlin (October)
- Lean Kanban France, Paris (November)
