Xcode, Git Bisect and You

Have you ever been in a situation where you’ve had a bug in your application and you had to find a commit where the regression was introduced? Do you get chills just thinking about how laborious this is? This is a common occurrence when working on a large project, with a lot developers. If this sounds familiar to you, you should definitely read on.

Example

Situations like this can be grueling. If you’re lucky you’ll find that faulty commit during the first 10 iterations. If not — you’ll spend a whole day checking versions and making manual tests. Sounds like fun?

Rather than operate in a purely theoretical world, let’s use an example project. We already have some commits:

git log --oneline
Output:
0aae197 Change #8
a36bda3 Change #7
f4bc78d Change #6
c320b43 Change #5
6a2fc42 Change #4
87b1295 This is not a commit you are looking for ✨
498883f Change #3
714a912 Change #2
8a91850 Change #1
519732f Add model.
301d978 Add empty project
a37bc3d Initial commit

At the center of you app you have a model to do those super important calculations for you:

Model 1.0

10 features later, at some point in time, a bug made its way into the codebase and users can observe a strange phenomenon where:

1 + 1 = 3

Manual Search

You could start with “Change #8” and go all the way to “Add model.”, but this is risky. Linear search has O(n) efficency in Big-O-notation. The worst case scenario is that you will check all the commits… what a waste!

You should use binary search instead. To do this you need a sorted collection. Thankfully, that’s the case. The bug was introduced at a certain point of time, so all of the changes after that are failing and all of the changes before that are working. Binary search cuts the range of possibilities in half, after every check. O( log(n)) means less work for you.

Still, performing this manually gives every programmer an itch under his/her skin — all this jumping could be done by a simple script. But wait! No need for that — your old friend git has your back!

Git Bisect

Git bisect is a tool that makes this kind of binary search easy for you. You kick off this process by typing this into the terminal:

git bisect start

Then, you have to point the first known bad and last known good commit. You do it like this:

git bisect bad           # current commit by default
git bisect good 519732f # hash of the "Add model." commit
Output:
Bisecting: 4 revisions left to test after this (roughly 2 steps)
[87b12952fd769a695c7835da2d297a3b332e2d1d] This is not a commit you are looking for ✨

You can mark branch, tag or hash, because all of them point to a concrete commit. After you have closed the range, the tool will automatically checkout the next commit to test. There is also a shortcut that lets you do the same in one line (master branch is in the HEAD of our repository):

git bisect start master 519732f
Output:
Bisecting: 4 revisions left to test after this (roughly 2 steps)
[87b12952fd769a695c7835da2d297a3b332e2d1d] This is not a commit you are looking for ✨

When it’s done, we’re automatically positioned in the middle of the range. Now, after you test if the current version is good or bad you just type:

git bisect bad # or git bisect good
Output:
Bisecting: 1 revision left to test after this (roughly 1 step)
[714a912f19da12aec91c3574a8f088d65b58ec3d] Change #2

When you run out of commits to check you will have your answer.

Automations

You could say:

“This is awesome! What more I could want from this great tool?”

Well… an automated test, for starters. To omit all that manual testing, you should write a unit or a UI test that reproduces the bug and can be run on any of these commits.

If you already have a failing test for this issue, this is good… and bad. Good, because you have help in place to automate your search. Bad, because you had a failing test on your master branch. You don’t want to find out about your bugs from users, if your IDE is able to help you.

More often than not, you don’t have test for the bug, so you need to write one (in reality, this is the hard part).

Jumping all over the repository prevents us from adding this test as a new commit. Fortunately, git has a very specific approach to time — we can change the history!

Git rebase in a nutshell.

Git Rebase

The rule of thumb for git rebase is very simple:

Don’t do a rebase on pushed branches shared between many people.

To continue we need to mirror master and create a temporary branch in the place where we will add the test:

git branch test master
git checkout -b temp 519732f
Output:
Switched to a new branch 'temp'

Finally we can add and commit check for the bug. 
Here is a test, as an example:

Test for the bug.

We add the commit on top of temp and then we change the history, ⏱!

git rebase temp test
Output:
First, rewinding head to replay your work on top of it...
Applying: Change #1
Applying: Change #2
Applying: Change #3
Applying: This is not a commit you are looking for ✨
Applying: Change #4
Applying: Change #5
Applying: Change #6
Applying: Change #7
Applying: Change #8

Voilà!

Git Bisect Run

Now let’s automate the hell out of our tools, git bisect has one more cool feature. You can use run to execute something and based on its output (presence of stderr to be exact) it will evaluate if the version is good or not. I have used flag “quiet” to suppress output from the xcodebuild, because it can spam your terminal.

git bisect start test temp
git bisect run xcodebuild test -quiet -project Bisect.xcodeproj -scheme Bisect -destination "platform=iOS Simulator,name=iPhone 7,OS=10.2"

After few runs of tests you should get:

2533a3e5abbb96e12a83582984c4adf121ca0de4 is the first bad commit
commit 2533a3e5abbb96e12a83582984c4adf121ca0de4
Author: Tomasz Bąk <t.bak@appnroll.com>
Date: Tue Feb 14 10:50:40 2017 +0100
This is not a commit you are looking for ✨
:040000 040000 6e7d8c27c383c26f5aded96f730f2a70bb3b1406 91f346cac8f6f2656c83bfa1d61d919200e20723 M Bisect
bisect run success

We have our culprit! Let’s check what has changed in the model:

Model 2.0 with new “features”

No surprise there. A pure function was broken by an implicit state of the object. Same old story.

Alternatives to Rebase

Our example is very simple but in the messy real world rebase could generate a lot of conflicts. We could also have a broken build in one of our past commits (you need a CI if this happens to you). Cases like this basically make git bisect run useless. Integration tests or very coupled system are situations where you are on your own but if a bug is in an isolated unit, there is still hope.

To circumvent this problem, you can create a new project with unit tests’ target only and reference classes that you want to test from there. Thanks to this, xcodebuild will fail only when its supposed to. Starting from a repository you are bisecting, you can then type:

xcodebuild test -quiet -project ../Test/Test.xcodeproj -scheme Test -destination "platform=iOS Simulator,name=iPhone 7,OS=10.2"

Summary

Git is a powerful tool and most programmers can’t imagine productive work without it. To know your tools, is definitely worthwhile. This is one of my favourite examples of working smart, versus working hard.

If there is one thing that you should take from this post, it is:

Take some time away from your work to learn about your tools. You’ll be surprised by how much they can do for you.

The project used as a playground for this post can be found here: https://github.com/tomaszbak-appnroll/git-bisect

Please feel free to share your experiences with us in the comments below or via social media (send us some photos or videos too), you can find us on Facebook, Twitter, Instagram and Pinterest, let’s connect!

To learn more about App’n’roll, take a look at out our website and our other posts. If you enjoyed reading this article, please click on the clapping hands and recommend it!

All images used are CC0 1.0 Universal (CC0 1.0).

References: