Disposable applications and the future-proofing myth
The mindset that many organisations take when delivering change is flawed. Software shouldn’t be built to last years and years. It shouldn’t be made ready for future reuse in ways we’ve attempted to foresee. Future-proofing is a myth.
Real, immediate business value is regularly delayed so money can be theoretically saved by building shared IT platforms ready for whatever might come. How many times is that money really saved?
Large COTS products are employed because features *might* be used in the future. Money is wasted paying for packages when only a small set of features are actually used. In these cases, how many times would a far simpler software solution have met the actual need quickly and effectively? (hint: almost every Data Lake or BPM integration ever)
In addition to falling into the ‘future proofing’ trap, sunk cost fallacy plays a large part in reinforcing this flawed approach.
The conventional ‘enterprise approach’
Having worked with a wide range of enterprise organisations, the conventional wisdom is to build large, flexible IT solutions that fit into an enterprise-wide vision. The idea of disposability is almost sinful. The aim is to benefit from economies of scale, reducing overall IT expenditure. It is to meet the needs of many different stakeholders and users. IT effectively forms Lego blocks in a Lego kit that is enterprise architecture (https://eavoices.com/2013/12/17/lego-an-enterprise-architecture-perspective/).
However, when these blocks are reused the usage is a bit different from the assumptions originally made. The building block forced into a different shape. It maintains all the assumptions from its original creation, correct or not, adds some more knowns and a few more assumptions yet.
The result is a great deal of unnecessary maintenance effort, spiraling design complexity (the subject of a future blog), unpaid technical debt, wasted time and money and capability build that may never be used. More importantly perhaps, it results in significant delays delivering targeted business value as we waste time building for a predicted future. In my experience, enterprises are very bad at tracking (or even acknowledging) loss from this form of delay.
Furthermore, this entire approach forms a vicious circle as the effort and money invested cause sunk cost fallacy to kick in hard.
So, what should the mindset be?
Creating small, disposable solutions which solve known problems, not assumed ones. Disposable components and applications can be easily decommissioned when requirements change. They can be replaced when new needs are discovered, benefiting from learnings since they were created. Dispose of that which is now unused and create only what is now required. Built like this it is necessary to invest less time, money and emotion, reducing the effects of sunk cost fallacy, enabling better decision making and realising value quickly.
This is not a unique concept, but it is one that goes against what most IT leaders have learned (and in many cases what entire IT departments are built upon); that we must create for reuse. Those I’ve seen cover the subject of disposability have usually directed it at engineers. Make code to be disposable. Decouple, reduce unit size, microservice, microservice, microservice.
The idea of disposable software extends beyond the engineer, beyond simple products and web applications, to software and solutions built by even the largest enterprises. I suggest that the ‘enterprise approach’ should be to understand and assess the ever-changing landscape of disposable applications and solutions and to apply hindsight to determine how and when to consolidate these to drive benefit for the organisation as a whole with certainty.
Central architecture functions and enterprise architects must take a fair proportion of the blame for failure to deviate from the current mindset. It’s a broad-brush statement, however I have rarely seen a group as stubborn and unquestioning of their own approaches, nor one that ignores criticism so easily. It is reminiscent of the opposition to evidence-based medicine, proposed by Archie Cochrane. Physicians still routinely ignore evidence in favour of their own cognitive biases and training, based on old or even non-existent evidence.
Disposability in action
An excellent example of disposability in action is at Uber. As a highly data-centric organisation, moving data around is a critical aspect of their business. Over time engineers have implemented multiple workflow systems, each designed to meet the need in-front of them, ensuring data flowed and that business was getting value from it quickly and effectively. At a point it became clear that the cost of running these separate solutions was becoming greater than the cost of consolidation. A team developed a consolidated workflow solution and decommissioned the existing ones. This new system itself was also mostly disposed of when it was rearchitected at a later point as needs evolved further. At each of these progressions prior learnings were used to replace what already existed highly effectively.
Why this subject?
Why did I write this? Recently I’ve been considering some of the big picture mistakes that are commonly made during software delivery and questioning conventional wisdom. As I did this, I came across shocking examples of waste due to ‘design for reuse’ behaviour. Doing this has caused me to think about my own decisions and to question my own mindset. Hopefully it will trigger the same introspection for you.
I’ll close on a shameless plug. I’ll be speaking (and encouraging discussion) at our own Velocity event, where I’ll be challenging the typical approach to data programmes within enterprise organisations, and discussing how disposability can apply here too.