How to pick a good branching strategy

It’s not so simple, so in this post I’ll be exploring the complex relationships between branching strategies, pull requests, and the factors involved in writing good commit messages.

Should we use the Conventional Commits standard for our commit messages, so we have better commit messages?

Well… that’s not an easy question to answer. Like so many other things in software development there will be no easy and universal yes or no for any practice. The right choice will always be contextual. And it may change over time.

As the Copyright Agent team has been growing, we have been having internal talks, trying to figure out how to build the most efficient team, while taking into account the many different backgrounds of the developers on our teams. Growing quickly means that we are putting all sorts of different practices into a giant melting pot. In order to work efficiently with each other, we need to agree on at least some practices.

With this in mind, we recently explored the complexities of something as simple as deciding on a commit message format — tying it into branching strategies, how to use pull requests, and even coding practices in general.

In this post I will briefly go over some of those complexities, to illustrate why easy answers are usually wrong, and to hopefully help other teams navigate similar choices in an informed manner: maybe I’ll point out a connection that wasn’t clear before, or introduce a practice that was so far unknown to you. In that case, this post has served its purpose.

Why do we have commits?

In my first job, our version control system was Subversion (a predecessor to git), and we used it mostly to share code with each other. We did not review or comment on each others’ code — we rarely even looked at it, unless we had trouble integrating our own code into it.

Today, the norm is using some hosted git service that emphasizes communication between team members when code is about to be integrated into the main branch. The Pull Request (PR) feature in Github automatically pops up when something is pushed to a branch, and encourages developers to start a conversation about the code they want to share. Other developers comment, more changes can be made to the branch, and finally it can be merged.

Commits let us break down code changes in smaller bits, that are easier to have a conversation about. Commits are conversational.

“I think you should change this thing.”
“Done!”
“Ah yes, I see your commit. Looks good!”

Purpose first

The trouble with deciding on a standard for how to write commits is that commits exist in this whole context: there are branches; and when we want to reintegrate those into the main branch, there are PRs; and while the PR is ongoing is when we look at commits — but we may also look back at commits later on to figure out why a certain bit of code looks as it does.

Commits exist in a complex context and may be used in many different ways. If we decide on a convention for how to write commits without considering how we want to concretely use them, we might decide on a convention that incentivizes us in a direction we did not intend.

For example, Conventional Commits are very structured, and propose some standardizes labels, like “feat:” for feature and “fix:” for bug fixes. But are we sure — in our organization — that we would want an entire feature to fit inside a commit? Does that fit how we talk about features?

Are we sure that we want to separate features and bug fixes into separate commits? What if we introduce a new feature that removes old buggy code — it’s technically in both categories.

I have worked in several places where conventions were adopted simply to provide a right answer. And I do get it: it feels nice to be doing the right thing.

“I have labeled my commit with one of the categories — I’m doing well!”

But I’m afraid these kinds of conventions offer mostly a false sense of competency. The hard things about writing good commits are the same hard things as in making good conversation. They are not easily captured in a convention. And if the convention becomes the bar we measure people by, they may forget that the point is communicating well with their team members.

Perhaps most importantly, we should ask: what do Conventional Commits add to our development experience? Are we getting value out of the effort we put in?

Answers to these questions will vary by context. But rather than just looking at what an organization is currently doing and fitting conventions to that, I would suggest a bit of wishful thinking.

Where would you like your organization to be? What kinds of practices would you like your organization to have?

If you’ve only worked in a very particular kind of environment it might be hard to imagine alternatives. So that’s what we need to do first: expand the imaginable. Let’s talk about branching strategies.

Branching strategies

When do you create a new branch? How long to you let that branch live for? What do you do, when you need to reintegrate the code on the branch with the code on the main branch?

When I talk to or train developers who are new in the industry, they often have a very set idea of what the right way to use git is — but rarely are those ideas exactly the same. These very set ideas are shaped by education or first jobs, and it makes sense to start with a narrow set of rules in order to learn how to use the tools.

But here’s the thing: the rules are made up. You get to remake them.

Now, I wouldn’t recommend just randomly trying out things. But have a think about how you would like communication to flow in your development team, what kinds of checks and balances you want on code before release, and how you want to be able to use commits after release. With these answers in mind, I’m sure you’ll be able to adjust some things in your use of branches in git — and some of them may even be improvements.

Here are some broad categories of branching strategies to inspire you. Even with these broad approaches there is plenty of room to experiment with the finer details.

Squash and rebase

Some large open-source projects like to keep their main code branch linear and easy to get an overview of. They do this by having each commit on the main branch being a clean, finished feature or bug fix.

But such commits aren’t really representative of how development happens in those projects.

Here’s how it normally goes: A developer has an idea, writes a bit of code, and submits it to a project as a PR. But then a seasoned maintainer of the project pops into the Pull Request and points out some issues, or places where the submitted code does not adhere to the standards of the project.

That’s fine — it’s part of normal development for open-source projects to onboard new developers by letting them go through the motions of refining a feature until it fits in the codebase neatly. More seasoned contributors will know the standards and submit code that’s close to what is expected, but even then there will usually be at least a little back-and-forth.

So the developer submits some more code. Followed by maybe some more comments. And then some more code.

Each bit of code submitted is a commit, and it has worked well as a conversation — but how do we go from there to a single commit on the main branch?

This is where the squash comes in. Basically, git supports turning several commits into a single one — and you get to write a new, nice, encapsulating commit message for all of that.

And the second part: rebasing is a code integration strategy that simply applies the changes in a commit on top of the latest commit in the main branch. That way, there is no need for an explicit merge, and the squashed commit is just placed as a neat next step on the main branch.

The downside is that you lose out on some of the details of the conversation that was being had while the code was being developed. There are ways to preserve the discussion, but not in a way where it will be directly visible in tools like git blame when inspecting code.

Many open-source projects would argue that context necessary to understanding code (such as might be found in a particular commit relating to a small change) should be capture in the code instead, as a comment or through better naming. That’s one way of mitigating the downside — but it requires more work and better discipline.

Github Flow

For a lot of people, this approach is what they think of as the normal way to use git: you create a new branch when you start working on a new feature; when you feel that the feature is ready, you share it with others through a PR; once done, the code is merged into the main branch of the project, preserving the commits that were made during development.

The immediate advantage is that the conversation that would be lost with a squash and rebase strategy is preserved. But it does introduce other difficulties.

Broken commits

What if, during development on a feature, you introduce a bug, that is then caught in review? Should the buggy commit be placed on the main branch?

This may seem innocuous, but allowing broken or non-compiling commits may make it almost impossible to use tools like git bisect, which would normally allow developers to identify the origin point of bugs that may otherwise be hard to trace down. (git bisect does a binary search through the code base’s history, applying some test at each point it looks at, quickly finding the exact commit where some behavior changed.)

Depending on your team and what you intend to use commits for — or might use them for in the future — it will be significant which standards you keep commits to. Are commits allowed to break code so it doesn’t compile or run? Even if the code compiles and runs, are commits allowed to introduce regressions, where something that used to work no longer does?

Once you’ve come to a conclusion on whether your commits should compile or not, there are more parameters to consider.

Messy reintegration

Just like with the squash and commit strategy, you could choose to avoid merge commits by rebasing each feature branch on the main branch before integrating it. This allows git to integrate the code with a so-called fast-forward, where it just places the commits from the feature branch on top of the main branch. Anyone who has been looking at a bit of commit history with merge commits knows how confusing they can be to read and understand — good thing that they can be avoided!

Another place where merge commits are common is in long-lived feature branches. The longer a feature branch goes on for, the more changes will happen to the main branch in the meanwhile.

If a long-lived feature branch does its own thing for its entire lifetime, without ever being brought up-to-date with whatever has happened on the main branch, it will probably be a nightmare to reintegrate it. There will simply be too many differences!

For this reason, teams using long-lived feature branches usually perform periodic merges of the main branch into the feature branch, so the feature branch is up-to-date. This creates a mess of merge requests which are hard to parse. The mess, in turn, can be resolved by rebasing the feature branch on the master branch, instead. But this creates difficulties if there are several people working on a single long-lived feature branch.

For teams with long-lived feature branches, the final reintegration of code becomes a really expensive hand-off requiring a lot of work. In fact, in their book Accelerate, Nicole Forsgren, Jez Humble and Gene Kim present research showing that integrating code often is correlated with being a high-performing team.

So I’ll pretty much unequivocally recommend keeping feature branches short-lived.

But that also means slicing features as thin as possible: what is the smallest possible value addition you can make?

How big?

Smaller features are easier to assess and discuss. The complexity of a system rises exponentially with its interacting parts, and so reducing the complexity of a change will have a huge effect on the difficulty of understanding it.

This leads to a natural question about commits: how big should they be? What should they contain? If smaller is easier to assess, this seems to indicate an approach for commits: make each commit as small as possible while still being coherent. A single logical change.

I often make commits that are just fixing indentation, or adding missing punctuation, or extracting a single bit of code into a function. That may seem exceedingly trivial, but the value comes in not having the indentation fix mixed in with a more difficult-to-understand change, distracting from the actual core of the problem.

Commits can be chained together to create an easy-to-follow story for the reviewers and assessors of the code.

But bear in mind that this is rarely how the sausage gets made. Code is written as part of a creative process, trying out several different things and testing the outcome, seeing what works and what doesn’t. How do we align the messy experience of writing code with the neat stories we would like to tell reviewers, in order to make it easy to assess the final product?

In fact, git has excellent support for this, with features like the -p flag for patch-adding, that lets you add only parts of files you have changed. Of course, this alone isn’t enough: most of all it requires building good habits around committing code, and that just takes time and practice.

Having a common standard for what quality is expected of commits will let the whole team work together.

If a commit comes in that does too much or is hard to follow, that is an excellent starting point for a conversation about what led to that particular commit, and how it could realistically have been done differently.

Those conversations are a kind of reinforcement mechanism that help build and sustain good habits.

Always-on-main

So high performing teams integrate often with the main branch, you say? Let me show you just how often I can integrate!

The fastest integration strategy does away with branches and pull requests altogether — you always push straight to the main branch.

To a lot of people who are used to feature branches and reviews that will sound really, really scary. (And it is definitely the most radical of the strategies I’m giving an overview of here.)

Let me try and justify it to you.

The basic observation starts like this: any code that has been written, but not yet delivered to customers to provide value, is a waste. Until it is delivered it has been cost without impact.

If we want to quickly get feedback on what we are building, it might make sense to just get it out there and see what people think about it. If anything is wrong, our integration approach allows us to incredibly quickly undo or change as needed.

This approach does leave us without some checks and balances that exist in the other two branching strategies: there is no review, so no other person has to check the code at integration time to make sure it is up to standard (and no person can block it from going out if they think it is bad). And it is a requirement that code compiles at each commit, if each commit is pushed through a continuous delivery pipeline.

On the other hand, proponents of this strategy would argue that catching things in review is too late anyway. You need better practices early on.

For example, you can replace knowledge sharing and input from colleagues at integration time with pair programming, where you’ll get input on your code every step of the way. There will be two pairs of eyes on anything that goes out. Two people signing off, they just did it before the integration happened.

Catch problems before inspection

Another check often done at integration time is testing that the program runs and performs as it should. But, proponents would argue, that’s a case for better automated testing and better support for manual testing — not pull requests.

In fact, the idea of using better practices to avoid catching problems on inspection isn’t new or unique to software development. In his 1982 book Out of the Crisis, W. Edwards Deming presented 12 points for making businesses more efficient. One of those points was:

Cease reliance on mass inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.

Even if you don’t push every commit to the main branch right after making it, this is good advice. One way to make PRs shorter lived (and therefore have fewer complications) is to make sure there is less to discuss. And I don’t mean by simply not pointing out problems — but by removing the problems before inspection time.

Good practices are required

Most teams won’t be at a point where they can just pick up the always-on-main branching strategy. They need skills and habits around automated testing, pair programming, and other practices that build trust in the codebase as well as the rest of the team — as well as practices specific to always-on-main development, like feature flagging work in progress things.

But perhaps it isn’t so outlandish a goal to strive for after all.

Maybe striving for it can be part of motivating your team to learn all of these practices, that are valuable to know and practice anyway.

It’s up to you

I’ve gone through some approaches to branching, pull requests and how to use commits. Hopefully that has given you something to think of. But ultimately it is up to the individual, the team, the department and the organization to decide what they want to strive for, which approach they think will be the best fit for them.

• Will you use branches at all? If you do, how do you ensure that they are as short-lived as possible, to avoid lengthy and difficult code integrations?
• If you use pull requests, how do you make sure to make them as efficient as possible for sharing information and knowledge; getting feedback; and building the best solution?
• Whether you use pull requests or not, how can you minimize the amount of errors found in inspection, so less time is used on back-and-forth?
• When you write commits, should each commit compile? Is it allowed to introduce regressions?
• How do you make as small and easily understandable commits as possible, in order to reduce complexity and help reviewers understand the changes you have made?

Maybe we should return to Conventional Commits for a brief final evaluation.

To me, the default suggested labels for commits seem to fit quite nicely with the kinds of commits created by squashing an entire feature branch into a single commit on the main branch, as seen in the squash and rebase branching strategy, I described above.

This also fits with other suggested labels, like marking commits that introduce breaking changes as such.

They seem to support the conversational use of commits less well. But, as a colleague pointed out, the labels may help some developers realize that they are doing too many things in a commit — and it is entirely possible for a team to adopt their own set of valid labels (although I do think that ends up stretching the original intent of the convention a bit).

In most cases, I would much prefer setting expectations (e.g. if more information than a brief title is needed to understand a commit, it should be provided in the commit message body) and establishing principles (e.g. a commit should always compile) rather than adopting conventions — especially in new and rapidly changing teams, where conventions may quickly become needlessly constricting.

At Copyright Agent, our current purpose is learning how to work efficiently together as a team, while everything around us keeps changing.

We’ll try our best, we’ll experiment, and we’ll keep adjusting.