Cynefin and Software Testing — The Complex Domain
The Complex domain seems to be where all the fun happens for software development. It also seems to be the point where we most often get into trouble. In the complex domain, there are many variables that we simply do not understand until after we have interacted with them. If the Obvious and Complicated domains are ‘machinery’ — you have known inputs, you crank a handle and what comes out is predictable to a large degree — then think of the complex space as ‘ecology’.
This leads me to a fantastic example of not understanding a complex space until after interacting with it. In the book Seeing Like a State, James Scott describes a number of abject failures due to oversimplification of a complex space. One that sticks with me is the failure of managed forests. These projects were set up to create a ready supply of timber by planting row after row of fast-growing trees, laid out with military precision. In short, these projects were a disaster and the crops were prone to disease. In hindsight it was apparent that because these crops did not have the benefit of the usual flora and fauna one finds where these trees naturally occur, the natural defences and mutually beneficial symbiotic relationships were also absent.
In effect, you can say that the complex domain can be better understood having tried something. You can forensically hunt down the elements of failure. The same is not true of success, as it’s difficult to know which variables to pay attention to and how they interact.
This is where we tend to find ourselves with software development though. There’s this sort of slightly dystopian doublethink where we try to build something that has not been built before and we plan for it to be done by a certain time and assume that we’ll get it right the first time and that it will cost a certain amount. Of course anyone actually building the thing will smirk or roll their eyes knowingly as the project gets bogged down, as people change their minds, or previously implicit requirements and emergent complexity become apparent.
Unless you’re building something completely trivial, or you’re doing a coding kata for the thousandth time, software development is a process of taking something with a large number of variables that interact in unknown ways, and acting on them to observe the results, understand emergent patterns of behaviour and to add constraints that ultimately make this behaviour predictable enough to exploit.
It is a given that you’re probably not going to get it right the first time. You may need to try a few different things before you understand what the right thing to build is. We see this with UX and user research folks doing paper prototyping with the customer. They run many rapid (often parallel), safe to fail experiments that are cheap and help move and idea with potential, to something demonstrable that has actual utility. It is incredibly valuable as it helps avoid premature convergence upon a suboptimal solution.
The same thing happens when you run spikes in development. There is usually some amount of complexity that is not well enough understood to act on and so one or more time boxed experiments happen to find out more. It may well be that what ultimately happens is different than the experiments run, but the act of experimenting purposefully allows you to better understand the system you are working in
Well functioning Agile crews do this in cycles. They define what they think is useful user value from a broader theme (epic), They (may) estimate what they believe is the complexity of the work they’re undertaking. If they can’t agree, they may run spikes. They implement stories and deal with emergent complexity — it’s understood that further complexity will emerge as they act on the space and that they have the ability to handle this. Once this piece of perceived value is built, they demonstrate it to the client who either accepts it or adds additional requests (sometimes both) and these are taken back into the bigger picture for the cycle to begin anew.
Honestly, I think where the software development industry gets itself into so much trouble is that they try treating complex spaces as complicated. The mindset seems to be that working software is predictable, so if we carefully plan to build predictable software, then the act of building it will also be predictable. We can get into the question of how we came to be here, but that is less interesting to me than helping software development professionals of all vocations recognise that software development is an activity that takes place in a complex space and we should be acting as such. Probe->Sense->Respond.
Originally published at http://testjutsu.com on August 2, 2017.
