Leading for a complex world

Effective leadership is critical to organizational success. However, what constitutes effective leadership is determined by how well leaders understand their roles and environments. As the world becomes increasingly complex, the nature of leadership must also evolve.

This is a series about how leaders can confront corporate challenges through the lens of complexity system science, a cutting-edge discipline that studies the convoluted exhibitions of the adaptive world from cells to societies.¹

In part 1, we introduce complex systems, outline their usefulness to leaders, and provide a few examples.

In part 2 and beyond, we suggest concepts, ideas, and tips for leaders to apply.

Mastering Corporate Complexity

“If you want to build a ship, don’t herd people together to collect wood, and don’t assign them tasks and work, but teach them to long for the endless immensity of the sea.” — Antoine de Saint-Exupery

Across the corporate spectrum we are witnessing a marshland of systems, continuously in flux and connected in odd yet intricately meaningful ways. The world is generating data at staggering speed and business operations struggle to cope as conditions surpass the cognitive limits of the world’s brightest humans. We aim to encourage and support corporate leaders by reviewing relevant business concepts such as leadership, vision, technology, etc. within a complex system frame of reference. We begin this series by outlining complex systems.

Making a Complex System Last

Complex organizational systems consist of a large number of components that interact in non-simple ways. This makes it difficult to interpret how components are bound together or predict the effects of their interactions on overall system behavior and performance.² People often attempt to comprehend systems with a lens based off their personal history and background (e.g. process-oriented if you’re managing an assembly line, expert-based if you’re a lawyer settling a complicated legal situation, etc.). However, approaching complex systems requires its own lens: a careful mix of conceptual application, testing, and diagnosis; intervention or even looking with the wrong lens may harm the system (the observer effect).³

We live in a world of corporate systems: many don’t last, but some do. The ones that do endure are able to withstand the scrutiny of the open markets and the ups and downs of economic cycles. A number of disciplines evoke this “will to survive” phenomenon: competitiveness in business, natural selection and Darwinism in biology, and hierarchy in social theory. Another one is strategic dominance, a concept from game theory.

Strategic dominance occurs when a system selects a set of strategies that is “better” than another set of strategies such that it “wins” across all future scenarios no matter how other systems strategize. In practice, successful strategies are better able to probe, sense, respond, and profit from changing opportunities and threats.

Strategies are fundamentally decisions about how to deploy resources with respect to tradeoffs.⁴,⁵On one hand, a system has a vision to work towards, and on the other hand, it has internal components negotiating for limited resources. A leader has to be continually scanning the environment for relevant market forces to adapt the system. Based on technically sound research and understanding, the leader can consciously pursue dominant strategies and refactor their systems to more optimal structures.

We can characterize a system in a variety of ways:

• visions and goals: “north stars” and preferred future states
• power: authority to stimulate or limit action or decisions
• trust: reliability and consistency
• knowledge: domain proficiency (skills and experience)
• agency: means (abilities, creativity, bandwidth, resources) to influence actions or decisions
• time: snapshot or change over a period of time

These characteristics can be graphically mapped⁶ by way of an organizational structure. We have highlighted three such structures — centralized, decentralized and distributed in Figure 1 (in reality this is more of a spectrum). The solid circles or “nodes” represent a record/entity/instance⁷ (can be any system component such as person, team, or stakeholder) and each link represents the interrelationship of two nodes based on a characteristic(s) at a certain point in time or duration. The most familiar characteristic that’s often mapped is power.

Fig 1. System structures (source: Hands-On Smart Contract Development with Hyperledger Fabric V2, O’Reilly)

For example, by mapping power and trust, a leader was able to identify that the finance team’s new budget approval process to the marketing team indicates a newfound lack of trust or grab for power. Similarly, by mapping knowledge and decision-making across teams, the organization learned that the product team was transferring important design choices to the marketing team during the high-pressure holiday season, thus illustrating that the strongest knowledge base and best practices were held by marketing.

In “human-centered” designed systems, people not only serve as the system’s nodes (e.g. social network like LinkedIn), but people’s feelings are also the end goal. Human-centered design is useful in situations where emotional input and aesthetic resonances are especially important (e.g. product development of a kitchen appliance that delights a home cook or service mapping of an airport for frequent business traveler comfort). However, in our experience, the human-centered design paradigm is not always appropriate or generalizable. In this article we suggest an alternative: “vision-centered system.”

Vision-centered systems are those where relationships, roles and responsibilities, coordination, and decision-making rules arise out of an overarching common purpose, i.e. vision takes precedence. This is directly inspired by ideas which argue that cogent visions lead to required changes and re-ordering, such as in social (e.g. from justice⁸ to policy making⁹ to democracy¹⁰), cyber-physical (e.g. Apple ecosystem), and techno-commercial (e.g. Italian haute couture crafts, Ikea, Tesla) systems through (linear, non-linear, and step) adjustments in a relatively efficient manner. After people buy into the system’s vision, actions and goals at all nodes cascade from the leaders who serve as torchbearers to the vision and its library of categories, canons, contexts, and catechisms. Companies with clear core ethos and an envisioned future are more likely to persist over the long-term.¹¹

In vision-centered systems, modern leaders will continue to confront hyperstructures¹², which comprise the hidden organizational mesh, layers, and levels among the more apparent and explicated nodes. In organizations, hyperstructures take on a variety of forms: gossip, legacy decisions, unofficial alliances & friendships, creative enclaves, groupthink behaviors¹³, divergent interpretation of internal goals or market forces, etc. that exert influence on explicit links, i.e. the visible structures, relationships, polices, procedures, etc. Systems that navigate the hidden realms can extract value and outwit systems that don’t.

To maintain a system state that is simultaneously authentic and productive, there needs to be some technique for simplifying complexity; acting randomly does not suffice. A method we propose is coarse-graining (CG).¹⁴ Coarse-grained descriptions smooth over finer details to derive a informative macro inference. Instead of waiting for “complete” information (which takes time to aggregate and sift through) to make a decision or analysis paralysis in the face of too much data, coarse-graining allows us to discern the system with “right-sized” data. For example, temperature is a coarse-graining of microscopic particles’ average kinetic energy. In most applications, a thermometer provides an accurate (and much quicker) method to predict a system’s state versus measuring the speed and position of all individual particles. So board members can disregard the speed and position of every particle in the conference room and instead use temperature to decide if a T-shirt or cardigan would be best. Applied similarly, a hospital’s new COO does not need to survey every doctor each moment to predict retention. Instead, a helpful coarse-graining to gauge employee morale could be counting the number of “friendly” discussions in the doctors’ lounge through weekly visits.

In business terms, this means that leaders ought to deploy top-down success criteria and institute necessary measurement processes. The intention from these is to simplify and derive sense of the various structural and dynamical metrics and behaviors and synthesize insights that can be actioned on meaningful time horizons. A word of caution: much like how machine learning engineers tune certain hyperparameters (parameters that are set as constants to define the model and are not learned through training on historical data) to optimize their models, leaders too must grasp and recognize the inter-dynamics of their organization’s variables and approach CG accordingly; gross over- or under-estimates of key variables can negatively influence assumptions and lead to unwise articulations of present states or incorrect predictions of future states. As always, double check what the input variables and output indicators are meant to represent or reveal about the system performance. In our previous example, the hospital COO may overestimate attrition by visiting the doctor’s lounge only on Fridays and miscategorizing heated (but ultimately healthy) end-of-week banter as employee dissatisfaction.

Fig 2. A leader’s power (Source: Leadership in Organization, Pearson)

As systems grow, they face greater risks and bigger threats. It is tempting to give up when faced with confusion and the foreboding feeling arising from the loss of control over our environment; lack of clarity easily wrecks our sense of purpose and unitive agency. However, we must keep calm and carry on with faith that vision, prescient planning, and continuous leadership development (traits and behavior) can unite and mobilize our joint efforts successfully, and that strategic dominance is indeed possible.

Scaling a System? Challenges!

“Our physical universe is bounded, both physically and in terms of possibility. Furthermore, this finite limit is true both in the near term, and in the indefinite future.” — Sandberg, Manheim

Scaling a system stretches and enlarges the space of possibilities, with more and more permutations or pathways available as a result of increasing dimensions: goals, time, people, data, etc. Scaling expands the amount of information that can be collected, stored, and acted upon exponentially as a consequence of different dimensions interacting in different ways. To successfully scale, we reiterate that leaders must adequately attend to and maintain the system’s core ethos (company DNA, first principles¹⁵, kernel, recursive index, etc.). Systems that keep the integrity of their products, services, and “promises” will ameliorate pressures and faults at system boundaries.

Systems contend with multiple kinds of challenges:

• information processing: limits on how much information a system can handle, which has consequences for tech/tool selection to extend system capacity¹⁶, knowledge making, resource management¹⁷ etc.
• path dependence¹⁸ and irreversibility: all systems have some built-in unidirectionality (sometimes known known as hysteresis¹⁹) which has consequences for irreversibility of certain actions, commitments and sequencing
• opportunities: decisions taken in regard to searching, developing, and utilizing resources, including talent and technology, curating public image, entering new channels and markets, raising/investing funds, etc.
• leverage points: identifying “extra sensitive” places in the system, where effective interventions will lead to proportionally larger improvements in system performance²⁰
• goals: setting up staggered goals such that achievement of one goal (or set of goals) boosts the charge and momentum for the next goal or set of goals (inspired by the space flight concept of gravitational slingshot or gravity assist maneuver²¹). We should also balance the trade-off between designing highly defined static goals versus relying on an overarching yet flexible vision to support exploration and reinvention.²²

System Structure Considerations

A system’s evolution and structural configuration depend on a slew of factors including, the environmental context or industry, the tasks and responsibilities of teams, signal/communication capabilities and technologies, competency or domain knowledge of agents, availability of resources, rule selection for decision-making and governance (e.g. by simple majority, by averaging results, by delegation) etc. Similarly, as structures change to better meet needs, we can measure the performance of systems in various ways, e.g. profitability, stakeholder morale, employee engagement, life expectancy of the system, etc.

While each system structure mentioned earlier — centralized, decentralized, and distributed — has the potential to be useful, structure choice is important because studies²³ on information aggregation indicate that the structure impacts the decision-making process and output. Centralized hierarchies, in which only a few individuals approve ideas, are typically more conservative, and accept fewer profitable ideas (more Type I errors or errors of omission) as well as fewer unprofitable ideas (less Type II errors or errors of commission). In decentralized polyarchies, in which many individuals must collaborate to approve ideas, the reverse happens: they accept a higher quantity of both profitable and unprofitable ideas. Hybrid structures — a mixture or cycling of hierarchy and polyarchy — perform the best overall in the rugged terrain of the business world. They do this by focusing on decision quality and improving the ability to learn, spot patterns, and explore & exploit resources (more in part 2 of this series).

Structures that are not centralized may have to rely more on deliberation²⁴ to shepherd system administration and management. Deliberation is policies and dialogues aimed at producing logically-reasoned, well-informed protocols and opinions in which participants revise attitudes and choices in light of evidence, new information, and discussions with fellow participants. Distributed structures with even minimal deliberation, such as a self-governing NFT community undergirded by blockchain technology, are able to make use of systemic rationality, i.e. collective wisdom, to increase cognitive sturdiness and thought diversity. These distributed structures can outperform individual articulated rationality²⁵ such as “expert opinion” or “special insight” as they better account for the revealed preferences and experiences of the many.

Due to the reduced role of a leader, distributed structures need to tackle reflexivity more head-on. Reflexivity is the capacity of a system to reconfigure itself in response to reflection on a variety of dimensions such as (1) sources of knowledge, (2) composition of system discourse, (3) system architecture, and (4) system dynamics. One framework used in studies is based on an evaluation of eight criteria²⁶, which are actually four pairs of system drivers bucketed in the dimensions mentioned before. In each pair, the two criteria are opposites of each other. For effective operation, the system needs to maintain a “balance” between each of these four driver pairs as appropriate to its vision, as depicted in Figure 3.

Figure 3. System dimensions for reflexivity: (i) Sources of knowledge: inclusivity vs. specialization (participation vs. expertise); (ii) Composition of system discourse: diversity vs. coherence (individuation vs. consensus); (iii) System architecture: accuracy vs. economy (poly-centric vs. centralization); (iv) System dynamics: flexibility vs. stability (continued scrutiny of buffered and rigid arrangements)

Deliberation is central to reflexive system learning because it can reconcile many claims internally and externally about its drivers. To determine whether or not balance is being maintained, the system needs to judge itself; the system must engage in reflection and reflective governance to deliberate and respond to questions asked by and about itself.

While braving real world challenges, empirical evidence shows that organizations cycle between extended periods of increasing differentiation (fission) and increasing integration (fusion).²⁷ Today, we are seeing genuine openness to alternative forms of organization such as, blended models, distributed decision-making, semi-autonomous work groups (self-managed teams²⁸, “teal organizations²⁹”), and composable XaaS (”anything as a service”) teams (e.g. security as a service) to unburden leaders, vitalize agency, localize processes, increase creative output, accelerate decision velocity, and enhance system cognition.³⁰

System Examples

For illustration of complex systems, we provide two well-studied examples:

A set of systems that exhibits all of our key concepts (vision-centered, seeking strategic dominance, maintaining core ethos, utilizing coarse-graining, practicing reflexivity, manifesting multiple structures, etc.) is the Command and Control (C2)³¹ system arising in a military context. In a C2 system the authority and direction over armed forces is exercised by a “commander.” Historically, “supreme commanders” like Caesar made use of purely centralized intelligence collection and decision-making authority to strategically dispatch Roman legions like pieces on a chessboard.

Fig. 4 Chess representation (source: ichess.net)

In WWII, Germany shocked France with its combined arms tactics — a concentration of various offensive weapons along a narrow front — also known as “blitzkrieg” (German for “lightning war”). Both countries had similar levels of technologies (such as the radio), but the Wehrmacht (armed forces of Nazi Germany) utilized the technology better by coordinating efforts and relaying decisions across disparate military branches: tanks, infantry, artillery, and aircraft. Eventually though, Germany was overthrown by the US who went even further to foster horizontal coordination and communication, and decentralize decision making to lower echelon commanders (either explicitly or by omission).

Post-WWII, various other C2 system structures emerged³² including the 1990s’ “network-centric warfare³³,” mediated mostly by a hub-and-spoke communication model. Many of these C2 leaders realized the need to develop cohesive doctrinal guidance and iron-clad operational procedures that designated clear responsibilities, unity of command (each subordinate in a formal organization should get an order from and report to only one superior), and quality communication between layers for faster action-reaction times in theater. While communications remained centralized (what was then technologically possible), initiative increased, the control line shifted forward, and responsivity improved.

Today, sustained progress in the fields of technology, operations, intel, and organizational behavior are converging to enable both widened cross-nation³⁴ and distributed C2 systems.³⁵ These newer forms of systems, augmented by AI products (e.g. Rebellion Defense³⁶), function as expansive, centerless, reconfigurable mesh webs that are more resistant and tensile.

Note, that across generations of warfighting, neither the overarching ethos nor goals for the system changed. Instead, the system evolved to more extensively and more quickly collect, affirm, coordinate, and analyze decisions (hence accelerating military objectives³⁷) penetrating smaller and smaller sub-systems. This fractal-like dispersion across the whole is a result of the generative interactions of a decomposable³⁸ core pattern. In system design, decomposability is when components that are highly interrelated are collated and grouped together to allow for intensive work. While sticking to the “original” basis for the C2 system, the military is now able to create more optimal representations and innovative strategic outcomes as a product of reflection, war gaming (adversary emulation, strategic simulation)³⁹, experimentation, and best practices.

Fig 5. Fractal representation (source: Dhanesh Budhrani)

Another example we present is the distributed system of traffic flow management in Mexico City.⁴⁰ Through the use of computer simulations that specialize in adaptation (instead of prediction which is computationally unfeasible), scientists working with traffic controllers are able to initiate self-organization (and shift human coordinators to less involved, “on-the-loop” roles) that significantly improves urban mobility.

Self-organizing traffic lights installed with edge sensors are constantly adapting to the changing traffic flow by dynamically modifying their signals to the “demand” of incoming cars. The traffic lights instruct the drivers what to do, but because of the sensors, the cars also feed forward their requests to the traffic lights. This loop promotes the efficient clustering of cars with superimposing trajectories into “green waves,” platoons of cars that roll through intersection after intersection without stopping. Note, traffic lights at one intersection don’t communicate with traffic lights elsewhere in the system and no one is programming a green wave based off of a predicted pattern. Instead, the green waves are triggered as an emergent, real-time property of self-organization. Idling time drops throughout the system as the only reason for cars to stop is to allow other cars to cross.

Most cities manage traffic using the traditional approach of programming traffic light signal timings purely based off historical density and configuration. However, traffic is a complex, multi-faceted problem. Using a distributed structure allows the system to unlock latent information (smart signaling decipher what will save time) and enable robust responsivity. The net result: a system that achieves more sophisticated collective goals including more efficient orchestration of traffic and reduced journey times.

Fig 6. Mexico traffic analysis (source: Data from Sky)

With complex system, structures and examples outlined, we will continue in part 2 to discuss the significance of leaders and what they can do in the face of mounting complexity and modern risks.

By Zainab Khan and Travis Vollmer

Resources:

1. Krakauer, David. “What is complex systems science?” (2019) https://www.santafe.edu/what-is-complex-systems-science
2. “Cynefin Framework” https://thecynefin.co/about-us/about-cynefin-framework/
3. “ Observer effect” https://en.wikipedia.org/wiki/Observer_effect
4. “Making Strategic Trade-offs” https://www.isc.hbs.edu/strategy/creating-a-successful-strategy/Pages/making-strategic-trade-offs.aspx
5. Seidler et al. “How to Make Strategic Trade-Offs” (2022) https://hbr.org/2022/06/how-to-make-strategic-trade-offs
6. “Graph Theory” https://en.wikipedia.org/wiki/Graph_theory
7. “Network Theory” https://en.wikipedia.org/wiki/Network_theory
8. Shenfield, Arthur. “Law Legislation And Liberty: Hayeks Completed Trilogy” (1980) https://fee.org/articles/law-legislation-and-liberty-hayeks-completed-trilogy/
9. Howlett et al. “Charles Lindblom is alive and well and living in punctuated equilibrium land” (2011) https://www.sfu.ca/~howlett/documents/Howlett and Migone — 2011 — Charles Lindblom is alive and well and living in p.pdf
10. Mantysal et al. “Planning as agonistic communication in a trading zone: Re examining Lindblom’s partisan mutual adjustment” (2011) https://www.researchgate.net/profile/Jonna-Kangasoja/publication/258178782_Urban_Planning_as_a_Trading_Zone/links/54169b6d0cf2788c4b35cabe/Urban-Planning-as-a-Trading-Zone.pdf
11. Stocker, Debbie. “Vision, values and purpose according to Collins and Porras” https://www.stockerpartnership.com/resources/articles/vision-values-and-purpose-according-to-collins-and-porras/
12. Norris et al. “Hypercomplexity” (2005) https://www.researchgate.net/publication/7197160_Hypercomplexity
13. Lehrer, Jonah. “Groupthink” (2012) https://www.newyorker.com/magazine/2012/01/30/groupthink
14. Flack, Jessica. “What scientific term or concept out to be more widely known?” (2017) https://www.edge.org/response-detail/27162
15. Musk, Elon. “The First Principles Method Explained by Elon Musk” https://www.youtube.com/watch?v=NV3sBlRgzTI
16. “Extended Mind Thesis” https://en.wikipedia.org/wiki/Extended_mind_thesis
17. Wolpert, David. “The concept of the extended phenotype provides a way to circumvent Landauer’s bound” (2020) https://www.santafe.edu/news-center/news/t-0290-david-wolpert-on-SARS-CoV-2-and-Landauer-s-bound
18. “Path Dependence” https://en.wikipedia.org/wiki/Path_dependence
19. Pruitt et al. “Social animals have tipping points, too” (2018) https://royalsocietypublishing.org/doi/10.1098/rspb.2018.1282
20. Meadows, Donella. “Leverage Points: Places to Intervene in a System” https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/
21. “A Gravity Assist Primer” https://solarsystem.nasa.gov/basics/primer/
22. Quattrone, Paolo. “Jesuits in the boardroom” (2022) https://aeon.co/essays/lessons-in-corporate-governance-from-the-jesuits
23. Healey et al. “Costs of collective wisdom: How resources influence information aggregation in organizational decision making” https://journals.sagepub.com/doi/pdf/10.1177/14761270211003849
24. Dryzek et al. “Deliberation as a catalyst for reflexive environmental governance” (2017) https://reader.elsevier.com/reader/sd/pii/S0921800916306619?token=C98AECF4A248313263AC2A50AA83722983586A29FE3BE1E0B30A0048CA1CDE414BE48E89EF9A4269E35BF569496557FF&originRegion=us-east-1&originCreation=20221113225729
25. Younkins, Edward. “Thomas Sowell’s Constrained Vision of Man and Society” (2001) http://www.quebecoislibre.org/010929-12.htm
26. Mertzani et al. “Metrics for Reflection in Distributed Information Processing’ (2022) http://abmhub.cs.ucl.ac.uk/2022/camera_ready/Mertzani_Pitt.pdf
27. Ethiraj et al. “Bounded Rationality and the Search for Organizational Architecture: An Evolutionary Perspective on the Design of Organizations and Their Evolvability” (2014) https://www.researchgate.net/publication/228172227_Bounded_Rationality_and_the_Search_for_Organizational_Architecture_An_Evolutionary_Perspective_on_the_Design_of_Organizations_and_Their_Evolvability
28. Yukl, Gary, “Leadership in Organization, Pearson” (2013) https://www.academia.edu/42939920/Leadership_in_Organizations_Global_Edition
29. “Teal Organization” https://en.wikipedia.org/wiki/Teal_organisation
30. Sutton, John. “Distributed Cognition” (2006) https://philpapers.org/archive/SUTDCD.pdf
31. “Command and Control” https://en.wikipedia.org/wiki/Command_and_control
32. Berg, Scott. “Introduction to C3 through comparative case analysis” (1990) https://core.ac.uk/download/pdf/36729982.pdf
33. “Network-centric Warfare” https://en.wikipedia.org/wiki/Network-centric_warfare
34. Demarest, Colin. “US, UK partner on command and control as Project Convergence wraps” (2022) https://www.defensenews.com/battlefield-tech/c2-comms/2022/11/22/us-uk-partner-on-command-and-control-as-project-convergence-wraps/
35. Brose, Christian. “The Kill Chain: Defending America in the Future of High-Tech Warfare” (2020) https://www.amazon.com/Kill-Chain-Defending-America-High-Tech/dp/031653353X
36. “Software transformation data into decision advantage” https://rebelliondefense.com/
37. “Kill chain” https://en.wikipedia.org/wiki/Kill_chain
38. Lamieri et al. “What is Near Decomposability” (2009) https://www.igi-global.com/dictionary/near-decomposability/19989
39. Mizokami, Kyle. “A Brief History of Naval Wargames” (2013) https://news.usni.org/2013/09/24/brief-history-naval-wargames
40. Ortega, Rodrigo. “Complexity Scientist Beats Traffic Jams Through Adaptation” (2020) https://www.quantamagazine.org/complexity-scientist-beats-traffic-jams-through-adaptation-20200928/