Navigating Change: Six years of technology consulting insights unveiled (part 1 of 3)

Having dedicated six years to IT and technology business consulting, addressing strategy, architecture, and operating model transformation, I’ve discerned a set of common patterns and guiding principles that have proven invaluable in my roles.

In essence, consulting and transformation revolve around key pillars:

1. Complexity: Our starting point with numerous interconnected elements, encompassing diverse cultures, organizations, geographies, and technologies. This intricate landscape presents challenges in understanding, prediction, control, and ultimately transformation.
2. Architecture: A discipline to manage complexity by cutting, structuring and reconnecting it into a more understandable and manageable system. A discipline to help deciding which pieces of lego to hold together.
3. Innovation: A discipline for navigating complexities to discover novel solutions, creatively combining elements to address problems.
4. Strategy: Charting a clear path forward, akin to crafting a plan for building a LEGO masterpiece.

In the upcoming section, we’ll delve into insightful principles and laws to enrich our consulting endeavors and drive meaningful transformations for our clients.

While my perspective primarily stems from IT, digital, and architectural viewpoints, it’s essential to note how these insights equally resonate in managing organizational complexity and driving transformational initiatives.

In this first part, we will explore the complexities of organizational and technology transformation and how architecture serves as a crucial framework for effective change.

I- Complexity

Most of our consulting work starts from complexity.

Complexity, in the context of IT and business, refers to the state or quality of being intricate or complicated.

Complexity arises from multiple factors that interact in varied and often unpredictable ways, making systems, projects, or organizational structures challenging to understand, manage, or change. The key dimensions that contribute to complexity being:

• Environmental Complexity covers external influences like market dynamics, regulations, and technological trends, complicating strategic planning due to the necessity to adapt to changes often outside an organization’s control.
• Technological Complexity involves the mix and sophistication of technologies increasing with the scale of systems, the interdependencies between them, and the pace of technological change.
• Organizational Complexity pertains to the structure and behavior of an organization, encompassing hierarchy, roles, responsibilities and decision-making processes, increasing with the organization’s size, diversity, sourcing model and geographic spread.
• Procedural Complexity deals with the methods and processes to complete tasks, especially in developing products, delivering services, or carrying out projects. It becomes more complex when you have to meet strict rules and standards, or when tasks require coordination across different departments within an organization
• Cultural Complexity concerns the diversity of values, beliefs, and behaviors affecting communication, collaboration, and problem-solving, significant in diverse or globally spread organizations, or those experiencing mergers.

Understanding the nature of complexity, its behavior, and strategies for managing it are crucial for developing effective approaches to manage transformation.

Successfully managing complexity typically involves decomposing systems into manageable parts, enhancing clear communication, nurturing adaptability, and promoting a culture of continuous learning and innovation.

This process demands a blend of analytical skills, to dissect and comprehend the intricate components and dynamics involved, alongside creative problem-solving abilities, to formulate strategies that either navigate through or simplify the complexities encountered.

Gall’s Law on simplicity “A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”

Gall’s Law advocates for starting simple and allowing for natural evolution to manage complexity effectively, ensuring systems remain functional, adaptable, and poised for continuous improvement.

Implication

1. Start with simplicity with a simple, working foundation before introducing complexity.
2. Evolution over Revolution through incremental development allows for adjustments based on feedback, minimizing the risks associated with complex deployments.
3. Architect for Evolution with modular, adaptable and scalable architecture to accommodate future growth and changes, avoiding the need for complete overhauls.
4. Avoid Overengineering to preserve the system’s ability to adapt and evolve. Prioritizing simplicity helps sidestep the challenges of complexity, such as elevated maintenance expenses and decreased flexibility..
5. Learn from Software Development with agile methodologies and MVP concept exemplifying Gall’s Law by focusing on iterative, feedback-driven evolution.

Glass’s complexity Law “For every 25% increase in problem complexity, there is a 100% increase in solution complexity.”

Glass’s Complexity Law serves as a reminder of the non-linear relationship between problem complexity and solution complexity.

Complexity is even more impacted by system organization. So complexity goes up as functionality increases, goes down as functionality is partitioned, but then goes up again as connections are made between systems. The overall complexity of a system is a delicate balance between all three of these factors.

Implications

1. Get clarity on the purpose, requirements and value to avoid building a complex system, organization or solution.
2. Optimize system organization to manage complexity partitioning and minimize unnecessary dependencies and interactions. Like entropy, complexity cannot be removed but it can be managed.
3. Balance Functionality and Complexity by evaluating how new features not just on its immediate value but also on how it affects the system’s overall complexity.

“In a distributed computer system, it is impossible to simultaneously achieve consistency, availability, and partition tolerance. You can only pick two.” Brewer’s CAP theorem

CAP theorem is a fundamental principle in distributed system that outlines the trade-offs between three key properties: consistency of the information in each node, availability of the system despite the failure of nodes and partition tolerance for the system to continue operating despite any number of communication breakdowns between nodes.

Implications:

• Complex organizations are like distributed systems with their operations and ressources being distributed and needed to communicate for sharing information (ex: prioritizing availability over consistency might be suitable for a service where most updated data isn’t as critical as ensuring the service is always online while financial operations might prioritize consistency to ensure that all transactions are accurately recorded and reflected)
• Making organizational or technological decisions is always a matter of priority and trade-offs. What matters when making a decision is to clearly understand the impacts on the various dimensions such as: performance, cost, scalability, resilience…

“Human minds navigate complexity linearly, while technology’s exponential growth intertwines disparate threads.” Ray Kurzweil

Technology advancements built upon each other at an accelerating rate led to increasingly complex systems and solutions. This is creating a cognitive dissonance for our brain which has evolved over millennia to process information in a step-by-step, linear, mono-threaded manner with limited short-term memory.

Implications

To keep pace with technology growth, develop skills and ways of working around.

1. Systems thinking and design thinking to enable a holistic understanding of complex systems and encourage innovative, user-centered solutions.
2. Continuous learning to keep up with technology and industry trends.
3. Technologies such as AI, automation and data analytics enhance efficiency and analytical capabilities for processing vast information quickly, automating routine tasks, and fostering digital collaboration across global teams.

“How do you eat an elephant? One bite at a time.”, saying

Proverbs used to convey the idea that large, complex problems or tasks can be managed and eventually overcome by breaking them down into smaller, more manageable parts instead of trying to solve it all at once, which can feel overwhelming or impossible.

Implications

Tackling large tasks “one bite at a time” is very similar to agile delivery and software architecture concepts by enabling:

• Modularity and the delivery of small value increments
• Adaptability and evolutionary design through rapid feedback and adaptation to change.
• Overcoming cognitive load overwhelming by splitting the complexity
• Achievement and Motivation by completing smaller tasks boosts motivation.

“Simple can be harder than complex; you have to work hard to get your thinking clean to make it simple.” Steve Jobs

“Simplicity is the most complex thing, but with simplicity you find the way to the heart and mind of your audience”. (Mozart)

Highlights the challenge of achieving simplicity and the significant effort to distill complex ideas into clear and concise concepts and ultimately, connecting with audiences on a deeper level.

Implications

1. Iterate until achieving clarity of thought and purpose, leveraging feedback and insights to simplify and streamline processes.
2. Adopt a persona-centric approach to designing solutions that align with users’ expectations.
3. Communicate complex ideas simply by distilling key messages and presenting them in a straightforward and understandable manner.
4. Embrace constraints as opportunities for creative problem-solving.

“An expert is a person that made all the mistakes that can be made in a very narrow field” Niels Bohr

Emphasizes that true expertise comes from overcoming challenges and learning from mistakes within a specific domain.

A widely concept proposed by psychologist K. Anders Ericsson called the “10,000-hour rule,” suggests that it takes approximately 10,000 hours of deliberate practice to achieve mastery in a particular domain.

Implications

• Experts increase the likelihood of success enabling early risk detection, adaptation to changes and maximization of opportunities.
• Mastery requires dedicated time and continuous learning.

II- Architecture

In the realm of organizational and technological transformation, architecture emerges as a powerful tool for navigating complexity and orchestrating change effectively. Architecture offers a systemic approach to making structural decisions, enabling organizations to navigate complexity effectively.

Whether it involves restructuring organizations or developing technological solutions, by embracing architecture, decision-makers gain a strategic framework to plan and implement transformation initiatives with coherence and purpose.

Beyond its conventional perception as a mere technical roadmap, architecture holds profound relevance across multifaceted domains. Its principles furnish a systematic approach to orchestrating and modeling complexity, furnishing a structured blueprint for transformational pursuits.

In this context,

• Software architecture principles, often overlooked as purely technical guidelines, hold immense value beyond the realm of code.
• Seminal concepts such as Conway’s Law, elucidating the profound interplay between organizational structure and system design.
• Additionally, architecture perspectives are crucial for cultivating solutions that resonate with diverse stakeholders.
• Decision-making serves as a core architectural discipline, guiding the direction, balancing trade-offs, managing complexity, enabling adaptability, and facilitating communication.

This section explores how these principles manifest in practical strategies for architectural transformation, both in the software world and the broader organizational context. By grounding our discussions in these foundational principles, we can chart a course towards successful and sustainable transformation in an increasingly complex and fast-paced world.

“The structure of software will mirror the structure of the organization that built it” Conway Law

Highlights how the architecture of systems developed by an organization inherently reflects its internal decision-making and communication structures.

This phenomenon explains the tendency towards application, data, and technology sprawl within large companies. In environments where avoiding the complexity, delay, and strategic misalignment associated with cross-departmental collaboration is a priority, it becomes more convenient to establish isolated silos.

The Reverse Conway Maneuver is a strategic approach to organizational design that seeks to optimize the development and management of software systems by aligning the organizational structure with the desired architectural outcomes. This might typically involve creating cross-functional teams aligned with the microservices they are responsible for, rather than organizing teams by function or technology specialization.

Implications

• Aligning team structures with software architecture goals can streamline development processes.
• Cross-functional teams are essential for reducing silos and fostering a cohesive software architecture.
• Organizational restructuring may be necessary to address inefficiencies in software development and architecture.
• Effective system implementation or transformation requires aligning organizational structure, strategy, and processes; otherwise, outdated practices may continue, and efforts to streamline IT systems could be hindered. Defining a new strategy without adjusting the organizational structure risks perpetuating outdated one.

“All models are wrong, but some are useful.”

Having a good model is key for having people understand a complex problem and guide collaborative problem-solving and decision-making.

The value of a model lies in its ability to provide actionable insights, making it crucial to select models that are fit for their intended purpose.

and so… “A model not used is probably wrong.”

Reflects on the practical side of model utility. Effective models must not only mirror the essential aspects of reality but also be accessible and meaningful to their users, ensuring they are integral to the decision-making process.

Implications

One of the core Architects added value is to support informed and collaborative decision making. Choosing practically applicable models is key.

• Stresses the value of simplicity and applicability in models to support effective planning and execution.
• Calls for regular updates and adaptations of models to maintain their relevance and utility.
• Advocates for collaborative model development, ensuring models meet the practical needs of decision-makers.

“Architecture represents the significant decisions that shape a system” Grady Booch

Architecture is defined by key decisions that dictate a system’s structure, functionality, and future adaptability.

These choices, concerning system boundaries, technologies, and integration, directly influence the system’s evolution, scalability, manageability, performance, and security.

Implication

• Underlines the critical role of informed, strategic architectural decisions in supporting an organization’s or system’s long-term growth and adaptability.
• Suggests the necessity of establishing a technology governance process to ensure the system’s enduring performance and alignment with organizational goals.

“Avoid One-Way Door Decisions” — Werner Vogels

The “One-Way Door” and “Two-Way Door” decision-making principles help guide architectural and strategic choices by categorizing them based on reversibility and commitment level.

“One-Way Door” decisions, which are significant and often irreversible, necessitating thorough analysis due to their long-term architectural impact. These decisions lock in a path with no easy return, demanding a careful, strategic approach.

“Two-Way Door Decisions” are reversible and low risk, encouraging agility and experimentation. They allow for quick iteration and adaptation, fostering a culture of innovation without fear of long-lasting negative consequences.

Implication:

• Emphasizes the importance of identifying the nature of decisions to prefer reversible paths when possible, enhancing flexibility and innovation.
• Advocates for a careful, strategic approach to irreversible decisions, ensuring they are made with thorough analysis and at the Last Responsible Moment to minimize negative consequences.

“Last Responsible Moment” (LRM),

This principle is rooted in Lean manufacturing and Agile development practices and has been applied to software architecture to enhance flexibility and minimize risk.

Advises delaying decisions until the latest point possible without losing the opportunity to choose. This approach allows for gathering maximum information, maintaining flexibility, and managing risks effectively. LRM encourages informed decision-making by keeping options open until a clear path forward emerges, ensuring choices are made with the best available knowledge while avoiding premature commitments.

LRM is not about procrastination but making strategic decisions at a time when you have the most information available while still being able to act effectively.

This approach encourages continuous learning and adaptation, which are key in fast-paced and uncertain environments.

Implication

• Promote a data driven decision making process evaluating options and impacts in their context.
• When building a system, focus first on core business logic deferring and abstracting the “technological details” such as middleware.

“Where you stand depends on where you sit”

Suggests that one’s perspective on an issue is influenced by their role, background, and stake in the outcome. It underscores the importance of considering diverse viewpoints in architectural decision-making to ensure comprehensive solutions.

implications:

• Incorporate diverse perspectives in decision-making.
• Consider a “personas” based approach to identify their different pain-points and perspectives.
• Identify bias

“Men do not stumble over mountains, but over molehills.”

Emphasizes the importance of attention to detail in software development but also any transformation where practical realities can lead to significant problems and impede the “best” plan.

This is also related to Amazon principle “Dive Deep. Leaders operate at all levels, stay connected to the details, audit frequently, and are skeptical when metrics and anecdote differ.”

Implications:

• Harness expertise to scrutinize and understand the finer details.
• Build a culture that values meticulousness, preventing minor errors from escalating.
• Use regular checks to catch and fix small problems early, combining data and real-world feedback.

“Program testing can be used to show the presence of bugs but not their absence!” Edsger Dijkstra

Testing, while necessary for identifying quality defect, cannot prove the absence of defects. Instead, improving software quality requires a focus on better development practices, such as thorough design reviews, pair programming, and adopting coding standards, rather than merely increasing the quantity of tests.

Implications:

• Recognizes that test (or overall metrics) addresses “known unknowns” (issues we’re aware could exist) but may miss “unknown unknowns” (issues we don’t anticipate), stressing the need for a deeper, preventive approach to quality.
• Illustrates a form of bias where there’s overreliance on testing (or overall metrics); assuming if tests pass, the software is flawless. This overlooks the complexity and unpredictability of software, underestimating the potential for unanticipated issues.
• Measuring things is the first step to start improving them but quality and performance goes beyond this shifting the focus to deeper practices and culture transformation…

“Work expands so as to fill the time available for its completion”. Parkinson’s law

Parkinson’s law is a reference to the self-satisfying uncontrolled growth of the bureaucratic apparatus in an organization, but the law could be generalized as “The demand upon a resource tends to expand to match the supply of the resource” with corollaries.

• Data expands to fill the space available for storage.
• A wardrobe will expand to fill all available closet space.
• If you wait until the last minute, it only takes a minute to do.
• Work contracts will fit in the time we give it.

An extension is often added: The reverse is not true 🙂

Implications:

• Set limits and constraints to prevent overconsumption
• Split available resources into smaller pieces (as practiced in Agile methodologies) to improve focus, increase flexibility and reduce risk.

“The number of transistors on a microchip double approximately every two years, though the cost of computers is halved” Moore’s Law

“Bandwidth grows at least three times faster than computer power.” Gilder’s law

While Moore’s Law focuses on the computational capacity and cost efficiency of microchips, Gilder’s Law addresses the growth of communication infrastructure’s capacity.

Both principles have profoundly shaped expectations in their fields, driving a marked reduction in the unit costs of computing and communication.

Implication:

• Technologies deemed impractical today may become viable within 5 to 10 years, owing to these ongoing advancements.
• The accelerating pace of network evolution enhances the feasibility of distributed architectures, including cloud computing and organizational structures, enriching supply chains and business models … also introducing increased complexity.

Part 1 Closing

In part one, we explored the complexities of organizational and technology transformation, focusing on two key aspects: complexity and architecture. We discussed how complexity manifests in IT and business, and how architecture serves as a crucial framework for effective change. Through examples like Gall’s Law and Glass’s Complexity Law, we learned the importance of simplicity and adaptability in navigating complex landscapes for sustainable transformation.

In part two, we will delve into strategy and innovation topics and explore their profound impact on decision-making processes and long-term perspectives. Strategy, as elucidated by thought leaders such as Michael Porter and Clausewitz, emphasizes the importance of prioritization, resilience, and strategic counteractions. We’ll examine concepts like Wardley Mapping, which offers insights into navigating complex business landscapes, and the significance of focusing on capabilities rather than solely on strategy for resilient architecture and efficient resource allocation.

On the innovation front, we’ll delve into insights from visionaries like Steve Jobs and Henry Ford, highlighting the role of creativity, curiosity, and challenging the status quo in fostering groundbreaking innovations. We’ll explore how embracing constraints, learning from failures, and fostering a culture of experimentation, as advocated by Jeff Bezos and Joi Ito, can drive continuous innovation and systemic change within organizations.

By dissecting these strategy and innovation principles, we aim to uncover actionable strategies that not only support informed decision-making but also foster a culture of adaptability and forward-thinking.

Related stories

Navigating Change: Six years of technology consulting insights unveiled (part 2 of 3) | by Tanguy NEU | Feb, 2024 | Medium

Navigating Change: Six years of technology consulting insights unveiled (part 3 of 3) | by Tanguy NEU | Feb, 2024 | Medium

Credits

Thanks to my 🤖Egyptian art type assistant for the illustrations.