Gitflow with Github and Cloud Build

What?

I want to implement Gitflow using Github and Cloud Build.

Why?

When developing software in a team, it’s a good idea to set some sort of standard process. This keeps things clear, ordered and helps the team to avoid procedural mistakes.

Also, if a new dev/devops/lead (or whatever) is added to the team, to have a well defined procedure improves the on-boarding process.

Gitflow is not the only flow out there, and your choice will depend on your project’s nature. But if you are building software that is explicitly versioned, or if you need to support multiple versions of your software in the wild, then git-flow may be a good choice. (more on GitFlow here)

I made this research when a client using Github+GCP (specifically GCP’s CloudBuild) needed to organize its repositories and processes. So this is the How to implement GitFlow using GitHub and GCP post.

How?

I will create a template repo with several pipeline files (cloudbuild.yaml files), for each part of the workflow. Then I’ll use triggers from Github to Cloud Build.

What is needed?

• A Github account
• A GCP account
• The will to do this 😉

Note

# 0

The pipelines here have dummy steps… meaning that some of them just print a string saying “Deploy steps”… but they are just templates, so you can then fill with your own actual test, check and deploy steps.

# 1

This is a WIP post, since I’m researching. I’ll improve this doc with new findings, so keep track of this and feel free to drop a message with recommendations or comments.

Steps

First we’ll set the repo. Then we’ll define a procedure to work with.

Setup the repo

Summary

These are the main steps:

• Start the git-flow
• Set Github repository (Settings)
• Use as base the repo template or add files to the current repo
• Create the triggers
• Grant IAM permissions to CloudBuild SA
• Take note on how to name your branches
• Enforce branch naming with git hooks

Start the Gitflow

Ensure you have downloaded (and added to you master branch) from here the following files:

• cloudbuild*yaml
• .githooks/

We’re starting with repos that have only a master branch. So we can start with the action stated in Extra Actions below to create the new develop branch:

git checkout master && git pull --rebase && git checkout -b develop && git push -u origin develop

Settings

Things to do in your repo before start working.

• Set develop as the default branch.
• Create an SSH key so pipelines can push, and add it to your secret manager. (as per this doc)
• Protect develop and master branch so nobody except the pipeline can push there.

Repo template

We need to add the basic files to the current repo (done previously). Files can be found here.

In the template repo there are these Cloud Build basic pipelines:

• cloudbuild-feature.yaml
• cloudbuild-pr.yaml
• cloudbuild-develop.yaml
• cloudbuild-master.yaml
• cloudbuild-master-deploy.yaml
• cloudbuild-release.yaml

Each one performs a part of the whole Gitflow process.

Triggers

Also, in the repo, is a triggers.yaml. With it triggers can be created like this:

gcloud beta builds triggers import --source=./triggers.yaml --project <project-name>

Specs about CloudBuild triggers.

* master tag must be defined, the proposed one fits this definition. Tagging must be a manual process since it triggers the deploy to prod. This kind of tagging allows us to more complex scenarios (e.g. v0.0.1 for prod, release/v0.0.1 for staging, etc)

IAM Permissions

Google Cloud Buid runs with a ServiceAccount named as: <project-number>@cloudbuild.gserviceaccount.com

This SA must have the following Roles:

Base: Cloud Build Service Account

If you needto deploy Cloud Functions in the project: Cloud Functions AdminService Account User

Access Secret Manager: Secret Manager Secret Accessor

How to name your branches

Prefix it with type, then a slash, ticket id and a brief description,(it’s important to follow these conventions since triggers rely on them) e.g.:

feature/TICKET-212-Add_new_provider 
fix/TICKET-312-Handle_None_case_in_var 
hotfix/TICKET-444-Fix_broken_endpoint 
release/v0.1

Enforce branch naming with git hooks

Option 1 — pre-commit

If the project uses pre-commit.

From repo get file .pre-commit-config.yaml, if you already have one append these lines at the end:

This hook will prevent commits to branches that do not apply to the regex.

To activate pre-commit hooks:

pre-commit install

Option 2 — plain git hooks

We will use git hooks to enforce branch naming.

The problem is… we can’t set local git hooks in an automated way. So, this will be a manual step that each dev must perform when cloning the repo.

In the same repo from which we got the pipeline there is a directory called .githooks/, there we stored the branch-naming-enforcement-hook... so each dev must run this in the local repo after cloning it:

git config core.hooksPath .githooks

The hooks has this code of pre-commit and pre-push files:

The procedure

Here are descriptions of the most common use cases:

• Clone a repo
• Work in a new feature or fix
• Release code to master
• Work in a hotfix

Clone a repo

After cloning a repo, un this command on the repo’s root dir:

pre-commit install
# If repo has no .pre-commit-config.yaml file, use plain hooks
# git config core.hooksPath .githooks

If you still do not have pre-commit installed read this.

Work in a new feature or fix

Checkout latest develop version:

git checkout develop && git pull –rebase

Create the feature/fix branch:

git checkout -b feature_branch_name
git push -u origin feature_branch_name

Create a WIP PR into develop branch to work on (maybe you won’t be able to create a PR until a change is submitted)

Work as usual (modify/add/commit/push) (on every push feature pipeline must succeed)

When done ask for approval and review (DO NOT MERGE)

When ready to merge add the following comment to the PR

/gcbrun

The develop-pr pipeline must succeed (it merges and deletes the feature branch)

Release code to master

Checkout latest develop version:
git checkout develop && git pull –rebase

Create the release:

git checkout -b release/v0.1.0
git push -u origin release/v0.1.0

Create a WIP PR into master branch to work on

Test it, if needed work in a fix (modify/add/commit/push)

When done ask for approval and review (DO NOT MERGE)

When ready to merge add the following comment to the PR

/gcbrun

The master-pr pipeline must succeed (it merges and deletes the feature branch into master and develop)

MANUAL STEP: Go to master and create a tag, it can be done in Github website or in the command line as follow:

git checkout master && git pull –rebase && git tag v0.1.0 && git push origin v0.1.0

The master-deploy pipeline must succeed (it deploys to prod)

Work in a hotfix

Checkout latest master version:

git checkout master && git pull –rebase

Create the hotfix branch:

git checkout -b hotfix_branch_name
git push -u origin hotfix_branch_name

Create a WIP PR into master branch to work on (maybe you won’t be able to create a PR until a change is submitted)

Work as usual (modify/add/commit/push) (on every push release-hotfix pipeline must succeed)

When done ask for approval and review (DO NOT MERGE)

When ready to merge add the following comment to the PR

/gcbrun

The master-pr pipeline must succeed (it merges and deletes the feature branch into master and develop)

MANUAL STEP: Go to master and create a tag, it can be done in Github website or in the command line as follow:

git checkout master && git pull –rebase && git tag v0.1.1 && git push origin v0.1.1

The master-deploy pipeline must succeed (it deploys to prod)

Extra actions

Add git-flow to existent repo

git checkout master && git pull --rebase && git checkout -b develop && git push -u origin develop

To Do

My to do list:

• hide “Merge” button on Github’s PRs
• protect branches

References

GitFlow

https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow

Git — git-flow mapping

https://gist.github.com/JamesMGreene/cdd0ac49f90c987e45ac

Cloud Build envvars

https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values

Cloud Build -Github authentication

https://cloud.google.com/cloud-build/docs/access-private-github-repos

Git Hooks

https://itnext.io/using-git-hooks-to-enforce-branch-naming-policy-ffd81fa01e5e

Originally published at http://juanmatiasdelacamara.wordpress.com on October 5, 2020.