Insights distill complexity to complication
In any problem space there is some mix of complication and complexity. Complication can be challenging, but it’s fundamentally concrete and tractable, something that a team can execute on. Complexity is, unit for unit, an order or two of magnitude more challenging because it fundamentally obscures what the right next step is.
You know you’re in the middle of a cloud of complexity to the extent that the next steps are not obvious — if progress isn’t being made and it’s unclear why. Common symptoms include people on the team starting to get confused, frustrated, and maybe even pointing fingers and shifting blame. These are the telltale signs that you’re dealing with ambiguity.
Complexity does not go away on its own. It must be vanquished: converted, with great effort, from its normal form to everyday, tractable, complication. An insight is a clarifying observation that distills some amount of what was intractable complexity to tractable complication.
A true insight causes a satisfying “aha!” moment of clarity, a satisfying pop as things click into place. We all hate complexity and will cling to any insights that seem to actually clarify things and reduce complexity. That means that a true insight is naturally viral and will spread within the team and even to neighboring teams. Conversely, if you have what feels like an insight but no one wants to share it, it likely isn’t actually an insight — it may be an observation, but it might not be a particularly clarifying one. True insights have the feeling of being discovered rather than created.
A good insight often feels obvious, but only in retrospect. They had to be discovered — unearthed, in some cases — at great cost. Insights may be small or they may be large, multi-faceted, and build on previous insights. An insight need not be concrete, per se. Sometimes they are a particularly apt metaphor. Sometimes they are a structuring or framing of a problem that helps separate extraneous factors from the primary dimensions at the core of the problem space. Sometimes an insight can literally just be succinctly capturing existing information in one well-organized document. Again, the mark of a true insight is that people want to share it and build on it.
Insight is hard. There are no one-size-fits-all patterns to do it. It’s about making connections between things: drawing parallels between neighboring problem spaces or historical examples, applying a new “lens” to the problem, taking a broader perspective, putting names to forces that intuitively feel like they’re lurking in the problem space in order to bring them into sharper focus. Insight fundamentally requires an open, inquisitive mind. One of the meta patterns of insight is creating the conditions under which collaborative debate can flourish. Collaborative debate helps turn the soil and uncover proto-insights. Those can then be refined and distilled into one- or two-page docs that capture the core essence of the insight more concretely and can be disseminated.
Complexity is such a frustrating state for a team to be in, and true insight is so hard, that sometimes it’s tempting to cheat. One popular technique is to choose a direction among any of the plausible ones and just start executing. This can be a reasonable strategy when the team is truly stuck going in circles, as a way to get some fresh air and possibly inspire the discovery of true insights. But if the complexity you’re dealing with is anything other than trivial in size, your random guess is simply a roll of the dice and is extremely unlikely to have been correct. It gives the illusion of movement, but in a way that likely doesn’t add up to anything long-term and can have enormous opportunity cost. This technique doesn’t actually reduce any complexity to complication — it just stashes it away under the rug, or sloshes it over to some other team (potentially to be sloshed right back in the near future). At some point that complexity will rear its ugly head again, and by then it may have grown and compounded. The only sustainable long-term strategy is to tackle the complexity, challenging as it may be.
That’s not to say you must resolve all complexity before you can execute, of course. Ultimately reducing complexity is not an end in itself; it is merely a means to enable higher-efficiency execution on your broader strategy.
One good technique for making progress in a complex space is to reduce the scope of complexity that you have to tackle. A particularly powerful type of insight in these cases is a framing of the problem that decomposes the problem space into a handful of discrete moving pieces. That allows you to identify the pieces that can be safely ignored for now, and allows you to focus on a smaller region of complexity.
Another way to execute even in the face of complexity is to identify concrete projects to tackle that are likely to be useful no matter what is lurking in that cloud of complexity. Maybe you can identify a piece of infrastructure that will certainly be required, or an incremental use case that extends an existing feature in the general direction you want to go.
No insight is perfect, and complexity is never totally vanquished in any given problem space. But if the total level of complexity is trending downward over time, you’re moving in the right direction.
PMs have many jobs, like setting a compelling vision, helping the team operate effectively, and ensuring the trains run on time. Reducing complexity over time by generating insights is another key PM job, but it’s abstract, hard to quantify, and easy to miss if you aren’t looking for it. I’ve found that the single, hard-to-pin-down thing that separates the PMs widely recognized as being truly great from those recognized as merely good is precisely this ability.
Note: The original version of this essay, confusingly, used the concepts of complexity for the easier type of difficulty, and ambiguity for the harder type, instead of complication vs complexity. I’ve updated the essay to use the word “complexity” in the sense it is used elsewhere, including the cynefin framework, complexity theory, etc.
