Why I love Trunk Based Development (or pushing straight to master)

15 min readAug 3, 2019

Summary

Developing using pull requests and feature branches, even when those are short lived, is easier for individual contributors but overall a suboptimal strategy for a team.

Teams that are using short-lived feature branches would usually benefit from aiming to work directly on master, since the practices that they need to have in place to be able to safely do trunk based development deliver numerous benefits: better collaboration, improved quality, collective ownership, and more.

Recap of the benefits of trunk based development over feature branches — The benefits of trunk based development

“No, we don’t push to master, are you out of your mind?”

Let me tell you a story.

I have recently joined a consultancy company. The fun part of being a consultant is that you get to work with a variety of clients, and indeed since I’ve started a couple of months ago I have worked with 3 different teams. For a bit of context, these teams were all co-located. Their size was around 6–8 people, made mostly of engineers plus a product owner, sometimes with a scrum master. Almost everyone in the team sits next to each other, with the odd work-from-home person from day to day.

When I join a team I always ask lots of questions, mainly to understand how they work and how I might help them. With all three of these teams, invariably we got to talking about version control and the conversation went something like this:

Me: “Ok, so what happens when I check in some code?”
Them: “You create a pull request, put the link in slack and someone will review it”
Me: “Oh, so you don’t push straight to master?”
Them: *stare at me with eyes wide open, like they’ve just seen an alien…a few seconds of silence…*
Them: *shaking heads disapprovingly* “No, we don’t push to master. Code needs to be reviewed first”
At this point I can see them thinking: “Is this guy serious? Is this the person that is supposed to “help us”?

Of course I don’t mind being questioned and people resisting my ideas, it’s part of my job. But what really surprised me was the incredible amount of resistance to the idea that someone might even consider pushing directly to master. I did a lot of googling and asked people in other companies, and it really seems like feature branches are the norm. It’s not even a question, it’s how our industry does things.

Coming from a company where instead pushing to master was the norm, I got thinking and decided to write this blog post. I want to explain why I believe that teams that are using short-lived feature branches would usually benefit from aiming to work directly on master, instead of using feature branches.

How teams use short-lived feature branches and pull requests

Let me explain what I observed to be the typical process of a team that uses short-lived feature branches:

The team has a backlog of stories to do. Typically they would have a “sprint backlog” with a bunch of stories to do in this sprint. (By the way, I have some ideas about how this could be done better, more on this in a later post.)
A developer is free, so they look in the backlog and pick a story. (The process of deciding *what* to pick varies, and can be a telling sign of a number of problems in the team. But again, more on this in a later post.)
The developer creates a new branch from master, and starts working on the story.
As they work, they commit and push code to the branch, at whatever frequency they are comfortable with. Could be minutes, hours, or days.
Whenever code is pushed, the build tool (Jenkins, circleci, etc.) runs a build on the branch. Typically the build compiles and packages the code, followed by running some tests. Possibly it might deploy to a dev environment.
When the development is complete, which could take from a few days up to around a week, the developer creates a pull request.
Someone in the team reviews the pull request. In some teams anyone can review it, but in other teams it has to be done by a tech leader or a senior developer.
The reviewer(s) might leave comments on the PR, asking for changes. In this case the developer goes back to implement those changes, then sends the PR again. This step could repeat a few times.
Eventually the PR is approved. The developer merges the branch into master, typically by squashing all commits from the branch into one big commit into master (“squash and merge”).
The CI tool runs a build on master. The pipeline to take those changes into production starts — sometimes they are deployed automatically all the way to production, sometimes they require manual approval at one or more stages.

Your exact workflow might vary, but I expect that this sounds familiar to a lot of people. This workflow is popular under the name of GitHub Flow.

Just to be clear, I know that there are other ways that teams use branches, for example Gitflow with long-lived branches. Those branching strategies open up a whole other world of problems. If you want to hear about those problems I recommend this presentation by Steve Smith. In this post however I want to focus on the problems that I see with short-lived feature branches, and why I think teams would benefit from trunk based development with no branches.

A successful alternative: Trunk Based Development, with no branches

What I suggest as an alternative is working directly on master, without the use of branches. Of course this has to be done in a safe way. In order to be able to do that a team would need these key practices in place:

1. Pair programming

You need to be doing pair programming, and a lot of it. By pairing with another person, the code that is being written is reviewed in real time, so there is no need for a pull request later on. On top of that, pair programming delivers an incredible number of benefits to a team: improved team resilience; people are less distracted; work gets done faster; higher quality; coding style becomes more standard across the team; people find better solutions to problems together; knowledge sharing; etc. (One day I’ll write about pair programming specifically and go into more details.)

In some situations you can even take this to the next level and do mob programming, with everyone working on the same thing, at the same time, on the same computer (e.g. for decisions that are better taken as a team, like architecture or design).

2. Have a build that you trust

You need to have a build that you trust. It needs to run enough automated tests to make you confident that the codebase is in a releasable state. And it needs to be fast enough that you can run the build locally before pushing to master, since you don’t want to push something broken that would block the rest of the team.

To achieve these two things, teams usually need to be following practices like TDD, BDD, and have an effective testing strategy — e.g. following the testing pyramid to minimise the amount of slow tests.

If the build is ever broken, fixing it immediately becomes the priority of the team. In the unlikely scenario that the fix was not quick, you should revert the broken changes.

3. Use “branch by abstraction” or feature flags to hide unfinished work

At any point in time there will be unfinished code in master. This causes no harm, since that code is not used in production paths yet. There are advantages however in having this code already in master: you notice integration problems faster, you can do more complex refactoring, and you can prove that the code will work by wiring it into your tests.

In practice, I’ve found that most of the times you can use the “branch by abstraction” pattern to avoid using code in production until it’s ready. For more complex scenarios I’ve used feature flags.

In my previous company working this way was the norm. I was there for nearly 6 years, but I know that they had been following these practices for well over 10 years. So I know for a fact that this way of working can be successful (Read the bonus part at the end of this article for more details of how we were working).

The many benefits of Trunk Based Development

I believe that the practices I’ve just described bring a plethora of benefits and healthy behaviours to a team, and should be something that teams aim for. Once you have them, you can safely push directly to master and forget about branches. Here are some of the benefits that I have observed, in no particular order (you might value some more than others, depending on your situation):

Earlier feedback: with a PR, feedback only comes when the developer thinks they are done. At this stage it’s usually too late to change anything substantial. When pairing instead feedback happens as the code is being written — or even before that, when you’re discussing how to approach the problem — so it’s much easier to change.
Better quality feedback: with a PR, feedback is usually given in a comment textbox. With rare exceptions, this is unlikely to convey the nuance that a discussion about software often needs. The original author naturally feels defensive about what they perceive as “their code”, and comments that easily come across as blunt have a strong tendency to cause annoyance or conflicts. When pairing instead you discuss your ideas face-to-face, where it’s an order or magnitude easier to clarify what you’re trying to say and have a healthy debate.
Collective code ownership: when code is written by just one individual there’s a strong tendency for that individual to see that as “my code”. You start hearing things like “oh, Alex wrote that, you need to ask them” or “we can’t work on this until Sam is back”. When pairing instead it’s much more likely that the team will build collective code ownership and see it as “our code”.
Team coding style: good teams write software that looks like it was written by the same person. It’s a sign that individuals value team work more than their own individual preferences. When pairing, it’s much easier to establish a team style, and to be always reminded of that.
More frequent integration (actual Continuous Integration): quite often a PR is only raised when a story/feature is complete. Pushing directly to master instead we’re integrating our code immediately, and we’re going back to the real meaning of “continuous” in Continuous Integration.
We get used to not breaking things: when we work directly on master we really don’t want to push anything broken, so we get used to running a build locally before pushing, and implementing code in a series of non-breaking changes. On the other hand, with branches it’s just too easy to ignore a failing build and only fix it at the end.
Easier to tackle large refactorings: when working in a branch we tend to dread doing things that are likely to cause merge conflicts, e.g. renaming packages, moving things around, architectural changes. Although still not easy, when working directly on master it’s at least easier to do those things, since we can commit small changes and have the rest of the team stay up to date. For really tricky changes, in my previous team we used to all gather around a single machine for a quick impromptu mob-programming session, where we all sit together and spike what the solution will look like.
More visibility of what everyone is working on: when changes are in a branch they are a lot less visible than if we all push to master. It becomes a lot easier to see what everyone is working on, and spot whether one of our teammates needs help.
Better tooling for reviewing changes: rather than looking at red/green lines on a web page, like people usually do when reviewing a PR, there are much better tools for reviewing what has changed before we commit it. We can use our IDE, or whatever tool we like.
Preserve the original commit history: when merging changes from a branch into master, often people squash all commits into one, losing the history of how and why the original author made those changes. When working on master instead we retain the full history of every single commit as it happened. I’ve worked in codebases that were 10 years old whose authors had long gone, and having the ability to use the git history to pinpoint exactly the commit where something changed was invaluable, as it gave us the chance to read the message that went with it and any other change that was done at the same time.

There are many more benefits that come from pair programming rather than from trunk based development, but I’ll talk about those in a dedicate post at some point.

Trunk Based Development is a sign of team health

Let me be very clear: trunk based development per se is nothing special and doesn’t give us any particular benefit. What does give us benefits is the practices that you need to have in place to be able to do trunk based development. For a team to be able to work directly on master, it means that they know how to work without breaking code, how to write good tests, how to work together effectively, etc.

You could say that trunk based development is an indicator of team health. And indeed, that’s what they found in the Accelerate book: after studying over 10,000 employees and 2,000 organisations, research has found that there is a strong correlation between a team doing trunk based development and that team being a high-performing one. In particular, they found that in high-performing teams branches lived for less than a day.

Optimise for team performance

When I ask teams why they use feature branches, or why they think you shouldn’t push directly onto master, I usually get answers like “We must keep master in a good state, it should always be releasable” or “We want to review each other’s code, to make sure it’s good quality and follows our standards”. I totally agree with both statements, and in this post I’ve shown you ways to achieve the same results without using branches.

However, there is another, often untold, reason why so many teams use feature branches: “It’s easier and more efficient if everyone works in their own branch, so we don’t tread on each other’s toes”.

Although that statement is true, I have to strongly disagree. This is where the root cause of so many team problems comes out. Most teams are optimising for individual performance rather than team performance. They look at how productive each individual is, rather than looking at how productive the team is. In Lean terms, it’s a perfect example of optimising for resource efficiency rather than for flow efficiency.

Feature branches optimise for individual performance, but trunk-based-development optimises for team performance. When you optimise for team performance, it will look like individuals are going slower. It’s a paradigm shift, and it’s counterintuitive. But it’s how you turn a group of individual contributors into a team.

Good use cases for feature branches

After spending so long telling you that you shouldn’t use feature branches, I feel like I need to clarify this: yes, there are some scenarios where feature branches are useful. As a rule of thumb, I would say that feature branches are good when the code has a clear owner, but someone else is doing the work.

The perfect example is the open source model: in a typical open source project you often have an owner, whether that’s a single person or a core team, and then a set of people contributing from all over the world, working in different timezones, without talking to each other. In this case it makes sense to ask contributors to send in a PR, since the owners would want to review it. It’s actually what GitHub invented PRs for! In this scenario, it’s also easier for the maintainers to reject any PR that they disagree with (e.g. if the PR was unsolicited).

A similar situation sometimes happens in companies that use an internal open source model: one team owns the code, but because they are too busy to work on it another team contributes to it by sending pull requests. It’s not the only solution to this problem, but it’s sometimes a good compromise.

“Ok, I’m convinced. How do I get there?”

If you’re currently using feature branches and would like to get to the stage where you can work directly on master, here are a few suggestions:

Review your testing strategy and work to have a stable build that you trust. It might mean that you need to start doing more TDD, to add more tests while keeping a fast build.
Start pairing more. If you don’t usually do much pairing, I suggest you initially start doing it on the difficult tasks, because it will be easier to convince someone in your team to pair with you. Then you can start doing it on more and more things, and build a core group of people in the team that like pairing. The rest of the team eventually will follow, especially if you establish a rule where code that was written when pairing doesn’t need a code review.

More Resources

Apart from the direct links in the article, these are good resources if you want to know more about the subject:

A special thank you to Will, Aram and Pritesh for their early feedback and for helping me develop and refine these ideas over the years. Also thank you to Jürgen Gmach for reviewing the draft and providing feedback.

Bonus: real life Trunk Based Development

Here is a breakdown of how my previous team was doing trunk based development and pushing straight to master. The context is very similar to the one of the teams described above: co-located team of 6–10 people; 4–6 developers, 1–2 tester, 1 business analyst, 1 team leader.

Developers pair full time.
One pair of developers picks up a new story. As part of the definition for getting ready for development, they write the skeleton of one or more acceptance tests, and a list of tasks for the story.
When the pair thinks that they are ready, they gather the team and show everyone the acceptance tests, with the intent of confirming that everyone has the same understanding of what the story is about and what the team is about to implement. This is called BDD, where we use examples to create shared understanding.
Each pair of developers picks a task from the story. Most of the times we had 2 or 3 pairs working on the same story, simply picking independent tasks.
Each pair commits and pushes straight to master, at a frequency that ranges from a few minutes to a few hours. They commit every time that they get to a green state: red-green-refactor, commit and push. We all loved small, frequent, commits, with clear messages indicating *why* we had just done a particular change. The commit message ideally uses business terms and focuses on the business outcome that we are trying to achieve with that commit. A good example is “Implemented the acc. test to prove that only a single customer can be allocated to a port”, or “Renamed class, since we discovered that the rest of the business refers to a ‘room’ as a ‘colo space’”.
Whenever developers pull from git they rebase in order to keep commits in the same order as they happened in real life.
Before a commit, the pair always runs a local build to ensure it’s successful. The build usually takes anywhere between 30 seconds and 2 minutes.
We followed the testing pyramid for our testing strategy, always preferring faster tests over slow ones. Our effort was greatly facilitated by the use of clean architecture, which allowed us to keep business logic isolated from technical details. It’s not the only way, but it worked great for us.
At any point in time the codebase is always releasable, by using the “branch by abstraction” pattern. In practice, being a Java project, we simply didn’t wire in the Spring beans until the very last minute. All the new code was available to test, but not used in production until we were ready to wire it in.
Whenever code is pushed, the CI tool builds it and runs all tests. We didn’t automatically deploy anywhere, but we could have done if we wanted to.
Development of a story usually was completed after a few days. At this point, we gathered around the team again to showcase what had just been implemented.
The latest version of the application would now be released into a staging environment, where our QA expert would perform some exploratory testing (typically lasting from a few hours to a few days).
If no surprises came up, the application was then promoted into production (typically the next day).
In the meantime, towards the end of the development of the previous story, one pair would have started looking at the next story to get it ready and keep a good flow. We used work-in-progress limits to keep this flow under control.
If at any point one pair felt the need to discuss something as a team (e.g. architectural decisions; design; trade-offs) they would simply ring a bell to grab everyone’s attention. All pairs would gather for an impromptu mob-programming session, spiking the solution out together. When everyone was happy the pairs would split again.