Michael Himbeault — https://www.flickr.com/photos/riebart/

FounderTherapy Engineering Guide: Code Management, Branching, and Deployment for Projects

At FounderTherapy, we have a set of standard practices that we employ when writing and managing code and deploying projects (usually to Heroku) that help us get our work done faster and better as a team. These are easy-to-learn and use techniques that can make collaborating on code and handing off responsibility manageable.

Master Branch Management

The master branch in git should be treated as the pre-production branch, to which all completed and tested feature branches are merged prior to deploying to the production branch.

No feature development or bug fixes should be made on master, with exceptions made only by the development team lead, and only when absolutely necessary. The master branch should always be completely build-able and deployable, and any code breakages on the master branch should be treated as emergencies that must be fixed before any development continues. It is the baseline branch, and as a result, should be treated as special. It is also the baseline branch from which all other branches are based.

Every developer should update their local copy of the master branch (and all other branches) by performing at least once per day:

git checkout master
git pull

This will ensure that all of the following branch management strategies are executed with a fully synchronized master branch.

Continuous Integration

CI for source code is handled by the service provider CircleCI. Every developer should create a login to CircleCI and should be invited to use the GitHub account.

All branches pushed to GitHub are watched by CircleCI, and any push to GitHub for any branch will initiate an integration test, installing a clean environment with all defined libraries and clean Postgres and Redis instances available. CircleCI will run all tests, and upon completion of the CI testing, will send an email and notify the “system-updates” Slack room with details of success or failure of the CI test.

If the test succeeds for either of 2 special branches (production and testing branches), CircleCI will also deploy the code to the appropriate Heroku environment. This process is detailed below.

Branch Merging Strategy

Whenever merging branches (for example from a feature branch to master), developers must always use the fast-forward only merging strategy:

git checkout master
git merge --ff-only {branch_name}

This is enforced so that a branch commits are applied cleanly and atomically together to the target branch. Merging this way also reduces the potential for code conflicts and keeps the branch history linear. If you want to default to --ff-only merges for a particular branch (like master and production) you can set it as a git default:

git config branch.master.mergeoptions "--ff-only"

We suggest that you get used to typing --ff-only all the time as it will ensure that the merge strategy is made explicit.

New Feature Development and Bug Fix Branch Development

When a new feature must be developed, or a bug fix must be made, the first step for a developer is to create a new branch from master:

git checkout master
git branch {branch_name}
git checkout {branch_name}

These steps ensure that all new branches start from the latest master.

Branches should be named as such: 123_feature_name, where 123 is the card number (in Trello) for that feature or bug, and feature_name is a simplified but understandable name for that feature or bug fix.

Each branch should be pushed up to origin (GitHub) once its created and regularly throughout development (at least once per day), to ensure that:

• Any ongoing development is backed up to GitHub
• Any inter-developer coordination for teams of two or more developers can happen through updated git branches

A GitHub Pull Request (PR) should be created for each branch to enable code review and feature tracking. The pull request URL should be added to the Trello card for that feature to make it easy for non-developers to follow along with the development effort. Using the merging strategy detailed below, PRs do not need to be explicitly closed as merging a feature to master will automatically close the PR on GitHub.

Working with Multiple Developers and Keeping Feature Branches Up to Date

Periodically (usually once per day, but the timing is developer dependent), the feature branch must be updated to incorporate any changes made to the master branch to ensure that the feature branch can be merged with master without conflicts. This is best done at the beginning of the day before any development occurs, to ensure that branches upon which multiple developers are working have the latest integrated changes from master and can update their local copies of the feature branch.

To incorporate master changes into a feature branch, developers should not just merge them into the feature branch. This can create invisible conflicts and lost code. It also makes future merges from the feature branch to master difficult and error prone.

The correct way to incorporate changes from the master branch to a feature branch is via rebase:

git checkout master
git pull
git checkout {branch_name}
git rebase -i master
git push -f

This process will rebase the feature branch to master HEAD and will then force push that update to GitHub (necessary because unless there is no change in branch point, rebasing is a conflicting change.

Whenever a rebase happens, all developers working on the feature branch should delete their local version of that feature branch and recreate it from origin:

git checkout master
git branch -d {branch_name}
git pull
git checkout {branch_name}

If there are any changesets on the feature branch that haven’t been pushed to origin prior to taking these steps (see above regarding always pushing changes to GitHub), they will be deleted. To prevent this from happening, the developer can create a branch from the old feature branch to hold those changesets, and upon deleting the feature branch and recreating it, they can cherry-pick their changes:

git checkout {branch_name}
git branch {branch_name}_rebase_backup
git checkout master
git branch -d {branch_name}
git pull
git checkout {branch_name}
git log {branch_name}_rebase_backup
git cherry-pick {changeset_id}
git push

If there are any feature branches that are branched from other feature branches, they must also be rebased any time the dependent feature branch changes or is rebased. This is very similar to rebasing from master, but instead uses feature branch HEAD as the rebase target (after feature branch has been updated or rebased):

git checkout {branch_name}
git rebase -i {dependent_branch_name}
git push -f

Code Review

Prior to merging a feature branch to the testing branch, the feature branch should be code reviewed by another developer to ensure all development work is completed properly and completely. A GitHub Pull Request should be used for all code review, and line notes or general notes should identify the developer (via @ notation) that did the work so that they are notified about any comments left for them.

GitHub PRs automatically update based on commits and pushes to git origin, so the code review discussion is kept in sync with ongoing development.

Creating a Testing Branch for QA

When a feature is complete and ready for testing, it should be merged into the testing branch. The testing branch is one of 2 special branches (the other one being production) that is automatically deployed to Heroku by CircleCI if unit and integration CI tests pass.

The testing branch should be treated as ephemeral, which means that it may be deleted and recreated at any time. This quality is in place to ensure that any set of new features/bug fixes are able to be cleanly tested, and to ensure that if any feature that doesn’t pass testing can be held back for more development work and isn’t merged to master before QA signs off on it.

The easiest process for testing a feature occurs when the testing branch is outdated or doesn’t exist. To get a feature into testing in this instance, the testing branch should be deleted on origin and recreated, and the feature to be tested should be merged into testing:

git checkout master
git branch -d testing # if necessary
git push origin :testing # if necessary
git pull # to get the latest master
git branch testing
git checkout {branch_name}
git branch {branch_name}_testing
# this temporary local branch is used to
# generate a clean merge line for the
# feature branch
git rebase -i testing
git checkout testing
git merge — ff-only {branch_name}_testing
git push

Steps 6–11 should be repeated for any other features that should be integrated into the testing branch.

If a testing branch already exists, and is current (meaning that it is a recent branch of master and the set of features merged into the testing branch are all currently in QA, the feature branch to be tested should just be merged to the testing branch without deleting it first:

git checkout testing
git pull
git checkout {branch_name}
git branch {branch_name}_testing
# this temporary local branch is used to
# generate a clean merge line for the
# feature branch
git checkout {branch_name}_testing
git rebase -i testing
git checkout testing
git merge --ff-only {branch_name}_testing
git push

CircleCI Deployment

CircleCI will run a full set of unit and integration tests on any branch that is pushed to GitHub. It will also automatically deploy both testing and production branches whenever new changesets are pushed to them on GitHub.

The configuration for CircleCI Heroku deployment is maintained in the file circleci.yml at the root of the git source tree:

deployment:
  qa:
    branch: testing
    heroku:
      appname: environment-qa
  production:
    branch: production
    heroku:
      appname: environment-prod

database:
  override:
    - cp config/database.yml.ci \
      config/database.yml
    - psql -U ubuntu circle_test < \
      db/structure.sql

CircleCI handles rebases and the removal and re-creation of branches appropriately, so the ephemeral nature of the testing branch presents no issue for CircleCI (it will handle newly created testing branches properly on its own).

Merging to Master

Once a feature is tested and QA has signed off on it, that feature can be merged to master. To accomplish this, the feature branch should be rebased from master and merged cleanly into the master branch:

git checkout master
git pull
git checkout {branch_name}
git rebase -i master
git push -f
git checkout master
git merge --ff-only {branch_name}
git push # this should only be done once

Steps 1-7 should be performed for each feature that is to be merged to master, and the final git push should be done after all branches are merged into the local master. Its a good idea to run tests locally before pushing to GitHub. Upon pushing the updated master branch to GitHub, CircleCI will run CI unit and integration tests. These tests must complete successfully before the master branch can be considered ready for production deployment.

Production Deployment

When one or more features have been merged to master, and a production deployment is ready, the proper process for deploying to production involves cleanly merging the updated master branch to production:

git checkout master
git pull
git checkout production
git pull
# if this fails to happen cleanly, then
# there are bigger issues with
# production diverging from master
# (or vice versa)
git merge --ff-only master
git push

If step 5 completes successfully, step 6 will initiate a final set of CI unit tests and then a deployment to the production environment will be handled by CircleCI.

If step 5 fails to complete successfully, that means that master isn’t in line with production. In this case, the developer must check the two branches to see where the divergence occurred.

If it is determined that production is correct and there was incorrect merging done on the master branch, those changesets must be reset and the HEAD of master must be rolled back to a known good version. At that point, all new feature branches can be cleanly merged into master and a merge from master to production can be attempted again.

If it is determined that master is correct, the production branch can be deleted and recreated from master:

git checkout master
git branch -d production
git push origin :production
git branch production
git checkout production
git push

Deleting Feature Branches

When a feature branch has been merged to master, it is no longer necessary to maintain (though it should be retained for a short period of time, usually a few days at least). At this point, the feature branch should be deleted and any dependent branches should be rebased onto master:

git checkout master
git branch -d {branch_name}
git push origin :{branch_name}