I remember being struck by one of the UK’s Government Delivery Service (GDS) mantras:
“The Unit of Delivery is the Team” — GDS
Indeed, along the same lines, Redgate, where I’ve worked for the last year, has one of its values as:
“We do our best work in teams” — Redgate, Our Values
The idea behind both of these is that software systems (amongst other things) is best produced by a team, not by individual contributors ploughing their own furrow. It’s a team that delivers software, not an individual ‘superstar’ developer.
Partly this is because of the vast array of different skills required to be effective, even at a relatively small scale of solution. And partly because of the classic Silo problem — if people on the same team aren’t talking to each other enough, the solution isn’t likely to work well as a whole.
There are different levels of effectiveness a team can operate at; Katzenbach & Smith (1993) describe Working Groups, Pseudo-Teams, Potential Teams, Real Team and High Performing Teams. The curve of performance for these has a dip reminiscent of the Tuckman model of “Forming, Storming, Norming, Performing”.
A High Performing Team is where all the skills of the team are synergistic, they are not a number of individuals doing their own thing.
It is also important to note, that you meet people where they are; the techniques described here are radically different, indeed heretical from accepted practice 20–25 years ago. Some will take to them like a fish to water, others will need support and guidance. And that’s not just the managers giving up what they consider to be “Control”.
Practicing Continuous Integration (CI) is where you prove software is capable of being built after every single commit (or at least once a day) by building and testing it after every commit (or push, in the era of git).
CI became mainstream and widely adopted over a decade ago, Continuous Delivery, now also about ten years old, is the natural conclusion of that train of thought.
Doing Continuous Delivery (CD) says that you should be able to ship your software after every single commit (or at least once a day) and that automated processes should be able to validate that that is so.
For completeness, Continuous Deployment (confusingly another ‘CD’, but not referred to like that here) is actually deploying those changes every change. This can be very powerful, but is not always appropriate, as you need to make sure you don’t just annoy your users (eg constant ‘Do you want to update’ reminders) and that it does not disrupt them.
With the explicit limit of changes taking no more than a day, there is a natural emphasis on making with any changes you do make are small. This is a Good Thing:
“Software has DISeconomies of scale” — Allan Kelly
Allan Kelly talks of software having diseconomies of scale, and this is because the larger the change, the longer it takes, with complexity increasingly exponentially, the more painful the integration with other peoples changes, and the more likely it is that they will be incompatible, so the larger the risk of breaking new or existing functionality.
At the end of last year, not long after I had joined Redgate, we were putting together a ‘Team Charter’ for the team I was going to be heading up this year. One of the central aims I had was expressed in a line in that charter —
“To be as close to Continuous Delivery as possible”
There is also an important point here around not trying to have the perfect CD ‘pipeline’ from the beginning, but ‘automate all the things’ over time. Have a ‘Minimum Viable Pipeline’ to begin with and build from there.
I’ve recently read the original Continuous Delivery book by Dave Farley and Jez Humble. It was published in 2010 so in many of the specific tools they discuss (this was before git was ubiquitous) it feels dated, but much of it is timeless. It is well worth a read.
Trunk Based Development
Trunk Based Development (TBD) is implied by Continuous Integration; changes should be Integrated (both ways) at least once a day. This precludes branches lasting more than a day, and certainly ones lasting weeks or months. The process is called Continuous Integration, not Occaisional Integration.
Just because branching is easy in git, it doesn’t mean you should. Like its name implies, it will betray you.
Not long before I interviewed at Redgate, I had an interview at one of the technology ‘Unicorns’ in Cambridge. I was asked to give a 20–30 minute presentation, and a surprisingly large number of people turned up for it. I gave the presentation I had given at Agile Manchester in 2016, which I had called Keeping Code Agile.
One of the (minimalist) slides said, quite simply, “Minimise Branching”.
When I put this up there was an audible intake of breath in the room.
I answered a few questions on this, but it was continually referred to in the rest of the interview process, so clearly hadn’t gone down well. At one point it was made clear that I obviously didn’t understand that this was not going to work when you had 70–80 developers*.
I didn’t get the job.
*= as an aside, there’s an interesting point made by Kevlin Henney that 5-10 developers will get the job done, 100 developers (working individually) will make a codebase big enough for 100 developers to work on. They may also get the job done. Eventually. Or not. Think diseconomies of scale again.
I actually know quite a few people that work there, and I met a good friend for lunch some time later. They said to me that I seemed to have hit a nerve, but Trunk Based Development (which I’d effectively described) was the only practice from Accelerate that the company didn’t follow. This was the first mention of the Accelerate book I’d heard.
This led naturally (thought not causally) into the work we’ve been doing at Redgate on Accelerate this year.
To me the central point is, and Accelerate makes this very clear, to be a High Performing Team (and why wouldn’t you want to be?) you need to be doing CD, and you need to be able to move submitted code to ‘shipped’ in half an hour — there isn’t time in that to do code reviews, or even overly-long builds you seem many teams suffocated by.
Work In Progress
The Team Charter also called out a Work In Progess (WIP) limit:
We will try to have a WIP limit of One
I could also express this as “We will try to do one thing at a time but do it well”.
As I outlined in my blog on Making Work Visible, having too much WIP is the ringleader of the Thieves of Time. The lack of focus, the context switching, the hidden dependencies and the neglected work all add up to sap your effectiveness.
As the author of ‘Good to Great’ put it:
If you have more than three Priorities, you have no Priorities — Jim Collins
And if you’re keeping WIP to one ‘thing’ you may as well be doing Trunk-Based Development…
And, if you’re doing Continuous Delivery, Trunk Based Development, and minimising Work In Progress, then you may as well be Mob Programming…
My team had been practicing Mob Programming since it came together at the beginning of this year, and it had been used in patches in other teams. In June I helped organise Woody Zuill to come in for a day’s workshop with a number of our teams who have now adopted the practice.
Either way, Mob Programming felt like a great step towards Continuous Delivery, allowing the team to focus on a single thing, get it delivered and then move on to the next.
To circle back to the beginning of the post, Mob Programming means that the team is working together, and is doing so closely, on a day to day basis. The closeness of the teamwork will draw the team together.
(I accept there is a chance that it could also blow a team apart, but that would probably have been inevitable anyway, this just quickens the process, flushing out a dysfunction)
There is literature that has been shown that Pair-Programming produces code with a lower defect-rate, and I would expect Mob Programming would be an extrapolation of that trend.
There are techniques which ensure that less-experienced members of a team are not just observers, but encouraged to contribute; alongside the Driver-Navigator model, it is suggested that if there are two ways suggested to solve a problem, that the suggestion by the less-experienced member is tried first. All of this works to improve the team as a unit.
As Woody Zuill explains, Mob Programming combines all the types of flow to make the team effective:
“Psychological Flow, Plus Team Flow, Plus Lean Flow” — Woody Zuill
Psychological Flow because context switching is reduced and team members can get in to ‘the zone’ with less risk of interruption; Team Flow because work flows through the team most effectively with everyone working on the same thing; and Lean Flow because Work In Progress in naturally limited to One.
Mob Programming really forges the links within a team, allowing a Brownian Motion of ideas being bounced around, refined and the potential for the very best solution to be found.
Just as Kent Beck describes the practices of Extreme Programming as mutually-reinforcing, it has become increasingly clear to me how much of a virtuous circle and self-reinforcing the practices I have outlined above are:
If you’re doing Mob Programming, you naturally have a WIP of ‘1’; all the team have been involved so there’s little point in pre-commit Code Reviews or indeed Pull Requests; which means branches are not required — Trunk Based Development. And Trunk Based Development feeds in to Continuous Delivery, so having automated the release process over time, when a change is committed (and pushed), assuming the tests pass* then the software is released.
With the team working closer together on a constant basis, the team is likely to improve over time, reinforcing the effectiveness of all the practices further.
But remember, this may make some feel uncomfortable, and like with any change, they will need support.
*= This can be a big assumption.