A return to pragmatism; Writing software is about writing software
A brief discussion on how we forgot to be pragmatic and how to pull ourselves out of the quagmire; scaling for the sake of scaling; why pi is 22/7(-ish); the meta and metameta of software development; a plan is good, but not as good something that works.
A pragmatic mindset is one of the most highly desirable traits in software development. Pragmatic engineers are uniquely focused on the task at hand and able to inform rapid feedback loops with the rest of their team, this means pointless work is dropped and ultimately products ship faster. This is obvious to most and ‘being pragmatic’ has been part of the meta of software development for decades. As the industry churns and (in some senses) evolves, this mindset is being dropped. This is because, and many do not want to believe it, the business of writing software involves primarily writing software.
There are many examples cited as reasons to shun pragmatism, although not as explicitly. We can take clumsy implementations of agile as a starting point, many of which bring supersized backlogs and overly detailed roadmaps. Huge assumptions are made about what will be happening and when, and often how: boring down to microscopic development detail in tickets, “this method now needs to take an additional parameter”, “add a log line on L54”. The authors of these plans have a foresight that rivals only the omniscient or perhaps just have an uncanny ability to be perceived as busy.
The flip side of this is the eye-roll inducing developers who cannot possibly start a ticket until every single detail has been explained, declaring that they are simply an extension of the machine itself and not yet capable of rational thought. How much is enough information to start on your project or ticket? The answer is ‘enough’, every case is entirely different and cannot be prescribed no matter how many agile seminars you attend.
Teams reinventing what already exists in-house is generally done for less-than pragmatic reasons. New CIs, new frameworks, new languages without a clear idea why. It appears as Whim Driven Development, usually that’s not the case, but rather a cognitive bias such as ‘this is what we used at my old place and it worked there’. I value experience, but experience with a reasoned argument even more. Organisations with fractured technologies like this struggle over time, and generally will tend towards championing a single technology, rewriting the old. This is usually because of the effects of hiring, in-house knowledge, differences in monitoring and alerting, etc. Entirely different stacks bring with them different challenges and suddenly you’ve got twice as many challenges.
I take the bastardisation of MVP is another example of deviation from pragmatism. The minimum viable product still seems like a Very Good Idea™, especially in companies arriving recently to a market. Through warping this, it seems that an MVP should not only be the first version of your product, but also one on which you must maintain very high quality, low time-to-market, and also have contain all features the C-suite executives spoke about last night. In addition, first-time users will need to literally fall in love with this product. It is worth managing our expectations of what is realistically possible at launch time, and also reserve our most human emotions for when is appropriate.
Early optimisation is another drift. While everyone appears to remember the Knuthiean quote, few seem to practice it. This can manifest at a code level where complex compression techniques are introduced way before an apparent need is arrived at, or at a higher level, where an assumption is made that soon the flood gates will open. An example is through something like platform or architecture choices, many will often go for the Biggest Everything despite the huge maintenance or cost overhead. Your 15 users do not require 3 *m5.4xlarges over 3 AZs over 2 regions. The law of large numbers has not been arrived at, the product theory has not reached reality: if a great number of users is reached the product will be completely different to what you have today.
When making decisions in this area, it is worth taking the pragmatic approach that, as of right now, and probably for the next two years, all swans are white. Other colours may eventually arrive, and when that happens the system will likely collapse (things break, don’t be surprised by this please), but operating under any other pretence it is unlikely that likely that you’ll have the luxury of ‘two years time’.
Taking architecture diagrams as the next stop, the tendency towards overly detailed mappings and structure is unnecessary. Most developers do not need to know which CPU register to update and when. Diagrams should be as low-level as required and taken lightly, they will always be incorrect either by a lot or a little. Drawing a mathematically perfect square in the real world is impossible, as is the circle, and right-angle. Architecture plans should be treated the same, as a theoretical blueprint that we don’t attempt to map entirely accurately into the real world as we know we can’t. Shortcuts and symbols should be used as with mathematical notation.
The perfect system does not exist in the real world, everything designed is incorrect, which, of course, doesn’t mean it shouldn’t be attempted. That knowing you don’t (and can’t) know everything is a good starting point for system design, it forces pragmatic decisions. In the real world, 22/7 is pi, or thereabouts, after-all, there aren’t any perfect circles.
MVP still means what you think it means, not what a consultant has decided will help increase their speaking fees. Small companies and teams should focus on building software, and not get overly bogged now in the meta surrounding it, analysing from this perspective is metameta software-development, it’s something that healthy pragmatic teams of engineers should be asking themselves constantly. Is our focus on writing and delivering software or on something external (or meta) to that process?