Git Bisect — Finding breaking commits
I have been working on a new project that involves working on a large mono-repo. When a pull request is made against the repository, a large number of unit and integration tests are run against the pull request — these tests can usually take anywhere from ten minutes to over 30 minutes to complete.
One of the most frustrating parts of this process is that some engineers have permission to land their pull requests without letting the integration tests complete (bad practice, but that is another problem). One Git feature that I have grown to love is git bisect, this subcommand helps you find bugs that were introduced into your project by performing a binary search over a number of Git commits.
How does git bisect find bad commits?
With git bisect you start by specifying a bad commit that contains the bug, e.g a failing unit/integration test. You then specify a good commit that does not have the failing test. Without a unit/integration test, git bisect cannot identify the bad commit.
Finding a bad commit
For demo purposes, you can clone [1] repository that has a simple unit test for the Fibonacci sequence. When you clone the repository, the commit HEAD contains broken changes that cause the unit test to fail, we can use git bisect to identify the commit that introduces the breaking change for our unit test.
First, let’s verify the unit test is failing on the HEAD of the repository.
Now that we have verified the HEAD of the repository is bad, we need to find a commit where the unit test was running successfully. Since we are dealing with a small repository we can check out the initial commit and run the unit test.
After checking out the initial commit, the unit test successfully runs. Now that we have our bad commit (HEAD) and our good commit (6dc3236976b) we can start using git bisect to identify what commit caused the unit test to fail.
Let us break down the steps we just ran. The first command git bisect start is required before we can state our good and bad commit. The second command, git bisect bad HEAD sets the HEAD of the repository to indicate a bad state since our unit test was failing. The third command, git bisect good 6dc3236976b specify the commit where our unit test was successful. After specifying the good and bad commits, Git performs a checkout of the commit that is in the middle of the good and bad commit. We then run the unit test on the current commit (85c501f4773) and can see that the unit test passes. Since the test passes, we indicate the current commit is good by running git bisect good. After indicating to git bisect that the commit was good, git checks out another commit (32197027bf9) where we will against run the unit tests, this time the unit tests fails so we indicate that the current commit is bad by running git bisect bad. After flagging the bad commit, git bisect returns the commit where the unit test first started to fail. We have successfully identified which commit caused the unit test to fail, and you can see in the commit we moved yield up one level causing the output to be incorrect.
Can we automate finding bad commits?
On a larger codebase, manually running tests and telling git bisect if the test was good or bad can be very tedious. However, git bisect has the ability to run a script or in our case run the unit test directly, the only requirement is that the script or the unit test provides an exit status of zero or one, with zero being good and one being bad.
The following commands show git bisect running our unit test directly and identifying the bad commit that was introduced.
