The Obsession with Dev-Fu and process puritanism
One thing I have recently realized is that we, as a profession, fall into the trap of focusing on how-we-do-our-work looks, instead of what we actually produce. It is reminiscent to Kung Fu and its emphasis on aesthetics, tradition and purity. We even sometimes add the suffix Fu, when we want to emphasize a person’s skill in something. E.g. Google-Fu or Git-Fu.
However, as learned from MMA, style and purity does not produce results. Pragmatism does.
“… the first UFCs were open to all and they were very much about putting style versus style….
…after a brief number of UFCs a pattern emerged. Grapplers owned strikers, who are mostly traditionalists and purists with no knowledge of grappling whatsoever. It was embarrassing for the traditionalists. Brazilian jiu-jitsu emerged from obscurity to become the most well-known and well-respected style that prepared people for MMA fights. Karate, Kung-Fu and other traditional arts were starting to be seen as inferior.”
This resulted in most of the new fighters to not do exclusively Karate, Kung-Fu or similar, but focus on the more pragmatic and “in-close-and-personal” style of Brazilian Jiu-Jitsu. It has evolved further since then. But the basic point, that was relearned, has been phrased well by Bruce Lee long ago.
So when the studies (in MMA = outcome of fights) shows that combining various fighting styles, appropriate for your unique situation and skillset, is a prerequisite for succeeding, then you ought to absorb it. And reject what is useless. It is not a magic recipe to make you win, but not following it will basically mean success is unobtainable.
For the last two decades a few similar, though much less violent, versions of this have occurred in the software development industry. We have seen it with the strive to agility (“there and back again”, to quote B. Baggins). And we now hopefully see the stirrings of a new hift. Or actually we should have seen such a shift a while back. But haven’t…
For quite a few years now, State of DevOps have been published and the findings have been repeatedly reconfirmed and we now actually have quite a strong foundation for knowing what actually drives software delivery performance.
Many of the prominent people in the industry echoes these findings. But software developers and organizations seem to have a bit of difficulty taking in the findings and conclusions and act accordingly.
In many ways and in many areas, we actually keep doing work in ways that are opposite of what the data tells us to do.
Even though it is called Computer Science or software engineering, we work more with Computer and software Gut-feeling.
We do what feels like it should work, emulate processes from places totally different and we, in general, are not good at actually letting data guide our decisions.
I have tried to delve into what data and studies are available on different subjects and tried to correlate this with my own experiences. I have selected a few of my pet peeves and strongest interests and will do deep dives on these.
First deep-dive: The lack of adoption of Trunk Based Development and non-blocking workflows
We might as well start out with the one that get people’s blood boiling the most. Even though our profession is not as violent as MMA, it is up there with relation to aggression and bile.
In “The DevOps Handbook” Jez Humble writes:
“Trunk-based development is likely the most controversial practice discussed in this book.”
And then goes on to describe the benefits provided by Trunk Based Development (TBD).
In State Of DevOps Report 2016 you can read:
“We have found that having branches or forks with very short lifetimes (less than a day) before being merged into trunk, and less than three active branches in total, are important aspects of continuous delivery, and all contribute to higher performance. So does merging code into trunk or master on a daily basis.”
In later work, this finding has been repeatedly verified and the case is stronger than ever. There seems to be no ambiguity.
Trunk Based Development (TBD) and continuouos integration improves software delivery performance.
And it is not just that it has a positive impact, it is an impact on the level of Monitoring, Deployment Automation and Code Maintainability.
Unless branches are kept at a minimum and are very short-lived, some improvements to software delivery performance remain unattainable.
And just to be clear. The delivery performance improvements shown in State of DevOps are not small… They are double-digit multipliers of improvement.
As a rule of thumb, the more branches you have and the longer they live, the more unattainable these software delivery performance improvements become. And most likely, software delivery performance is actually impeded.
Despite this, I have again and again seen people view “full”-feature branches, Git-flow, etc. combined with pull requests as the de facto standard in the industry. And challenging it can make people react like you are a heretic, complete idiot or simply not a professional developer.
My views on TBD (and related) are based on 10+ years of experience working with both TBD, feature branches, version branches and a mix of these. I have worked in several different domains and types of organizations, applications and systems. But primarily within full stack application and systems development.
I have also had a ton of discussions and done a lot of reading, trying to understand the pros and cons, what people hate and love, and also why feature branches and pull requests are so enticing. And why people choose that proces, despite what the data shows us. I have read more nasty discussion-threads about the subject than I think is healthy for most people.
Given my background, the following articles are primarily focused on software development using Git or similar VCS. Some points apply more broadly, but the articles are most relevant if that is the type of software you are working with. The process around the actually code-churn and feedback cycle, is not within scope of these articles.
For the sake of transparency, I can divulge that my preferred development setup is:
Feature toggle enabled Trunk based development in Mono(/big) Repository with post code integration reviews and continuous delivery to test environment.
A shorter (more catchy) name for this, is Trunk Based Development with Non-Blocking Review (TBD w. NBR).
The tale of two articles
There are basically two ways of trying to get a message across about why a behaviour should change. You can either flesh out the better alternative and why you think it provides better value. Or you can delve into why the current behaviour is problematic or providing less value and thus should change.
Being a bit greedy, I have chosen both of these approaches, by splitting my take on this subject into two different articles.
The first focuses on TBD w. NBR, trying to flesh out the how and the why of it. The rationale and the value.
The second article focuses on why I believe feature branches, pull requests and also poly repositories to be problematic and empeding performance.
Common for these two articles is the assumption that what we want to nurture in teams, to improve software delivery performance, are the practices and the culture we know to work.
- Continuous integration of work
- Frictionless flow of work
- Psychological safety
- Understanding the value chain
I.e. basically the findings from State of DevOps, Accelerate and research into High Performance Teams .
Thus evaluating whether to do something or not, should be evaluated on the impact on the goals above.
NB: I primarily focus on software development in businesses or other organizations. I.e. non-open-source contribution driven/volunteer software development.
I hope you read one or both of these articles, as they are intended to illuminate the issue from different angles. One highlighting the upsides to TBD w. NBR and the other spotlighting the downsides I see associated with and induced by feature branches, pull requests and polyrepos.
I look forward to what will, hopefully, be a pleasant and constructive discussion.
Part 1: TBD with NBR
You can read the in-depth article about TBD w. NBR here:
Optimizing the Software Development process for continuous integration and flow of work.
Part 2: The problem with feature branches & pull requests
For reasons how feature branches, pull requests and poly repositories works aginst the goals above, I have outlined various problems in this article:
How feature branches and pull requests work against best practice.
Next Deep Dive
If you have suggestions to what I could take on as next deep dive, I would appreciate the input. Maybe I will look into how best to ensure Continuous Quality (CQ) in the setup described (or any actually).
It can be based on manual or automated testing, monitoring and many other things. But in my view the reduction of batch size and instant feedback actually drives quality and value delivery much more than a high automated test coverage. And at a lower effort and opportunity cost. This indicates an interesting, and counterintuitive, equation and result. And investigating many of the assumptions related to automated testing and actual research into this can lead to some surprises.
Thanks to Antonio, Ebbe, Jonas and Andreas for feedback and discussions about this article:-)
There are many “opinion” pieces on TBD which I can recommend looking into.
Feature branches and Toggles in a Post-GitHub world
Why you should not use feature branches
Why Code Reviews Hurt Your Code Quality and Team Productivity
Continuous Integration and Feature Branching and Perceived Barriers to Trunk based development
Why I love Trunk Based Development
