The Destructive Pursuit of Certainty

We like certainty. We like to systematise: facts, processes, rules, explicit and detailed guidance on how to go about doing something. Like is probably the wrong term though, it’s more about comfort: cognitively stress-free or at least easier to cope with. Certainty too is questionable as a term, perhaps predictable is the better choice, maybe reliable.

At least in knowledge work, it sure seems like we want certainty. Consider a typical recruitment process:

  1. CV-based — full of what we know and don’t in the form of programming languages, algorithms, frameworks and such. Sprinkled with some examples of what we’ve done.
  2. Interviews — lots of tests to ensure you know your facts about this or that. It’s kinda like exams more often than not.

We want to know with confidence that you know stuff. There are other indicators of our desire for certainty too: the perpetual search for tools that reduce things to plug and play. Just follow the rules, plug in some details, all the complexity will be handled inside the product, don’t you worry. Ah, the silver bullet.

When I think about skill, I tend to view it from the perspective of the Dreyfus Model — it has issues, inevitably. Amongst other things it’s very black and white in its treatment of intuition vs analytic thinking. Perhaps it’s tinged with certainty? Eliminate the vagueness!

In spite of its weaknesses the general thrust that intuition features increasingly as competence grows fits well with my own observations at least. Good engineers have what I call “taste”, trading amongst constraints and facts using heuristics as much as analytical thinking. It shows up clearly in debugging hairy problems, where they take a leap in a direction that only in hindsight makes sense (the leap is validated by facts uncovered after the leap is taken).

What of creativity? Or innovation, if you prefer. There are lots of attempts to systematise the activity. There are rules for constructing the right environment, what processes to use etc. However, there is a substantial body of evidence that contradicts much of this advice. It suggests serendipity, intuition, taste, heuristics, unconstrained views and such are the source.

Facts, rules and processes can be found from a well curated library on demand. The cultivated application of heuristics, intuition and the like can only be developed through mindful doing. We have to fail, cut our fingers, sweat some, there are no shortcuts. Only from this can high skill and innovation be born.

We need to be nurtured too, during our labours. The path of rule and process is very narrow compared with the surrounding land of intuition. A coach, a mentor, a guide is not a “development opportunity” it’s essential to our growth and there’s so much ground to cover, we’ll be needing help for a long time.

Which worries me a great deal because when I look at recruitment, the obsession with systematisation (in some cases, mere ritualisation), the corporate mantra of uninterrupted, fast-paced work with little time for review and dialogue, I just see filtration. The selecting out of high skill and creativity. Yet in contradiction we say we want those things. Should I mention the complaints of a skills gap?

High expertise is what’s required to go after the big challenges, those earth-shattering leaps in progress, the big make-a-difference-to-humanity products. We and our organisations all claim to be intent on doing these things but it’s in opposition to our actions.

We’re wasting huge amounts of energy in our pursuit of certainty eradicating the very assets we need to achieve the big things we desire. The loss doesn’t stop there as, badly prepared, we fail to achieve our goals burning significant resources as we go. Failure is most assuredly a necessary part of learning but are we actually making any reasonable progress? Or do we simply buy some new tools, recruit some different people with the same old methods, adopt the latest process and go around again?

Some folks, some organisations don’t. They are the rare few, the ones that make the big things happen. I don’t see them going away but they will always be found in the dark corners, on the fringe, taking the longer orbit, off the beaten path. Care to join them?

“Genius is one percent inspiration, ninety nine percent perspiration”

— Edison

(Many focus on the 99% but “work hard & you’ll succeed” is doomed in absence of the 1%. Make sure that there are opportunities to shift focus back and forth serendipitously)

“competent practitioners usually know more than they can say. They exhibit a kind of knowing-in-practice, most of which is tacit. Nevertheless,”

— Donald A. Schön