Development with Git Hooks
Nowadays, having a decent CI/CD platform is a must. A robust CI/CD Pipeline provides a comprehensive set of tools that help your developers and engineers, in the context of both Dev & Ops, package and deliver software successfully to the end users.
Your Pipeline may consist of many tasks that gather around the ultimate CI /CD jobs — build, test and deploy. Depending on the pipeline, the automated structure of the pipelines directly provides a Rapid Application Development platform and lots of cool benefits such as ability to utilize Automated Testing tools and various deployment methods such as blue/green, canary and linear.
These automated tasks take a huge burden off developers’ shoulders when it comes to validation & verification of the implementations we’ve done. But sometimes these automated pre-merge checks are just not enough and some additional checks before submission of the implementation to the pipeline may come in really handy.
Recently, I’ve experienced a problem that made me go for a solution that involves such an additional check mechanism I described above. Let us discuss it below.
Problem
In Picus, we have a CI/CD pipeline and when it is time to merge any development branch these pipelines automatically run. Among many other tasks, our pipelines include a step for running our (any kind of) tests.
Trusting the pipelines, I was pushing my implementations incrementally to the remote repository and let the pipelines assess the code. It takes sometime for the pipelines to finish, so after waiting a while for them I saw that the tests running on the pipelines failed. After checking what went wrong in the tests I realized that I forgot to regenerate the mocks that we use in our tests 🤦.
I fixed that case at the time but this forgetfulness of mine kept happening again and again, making me waste tiny bit of a time on each occurence and making me go back and fix those tiny problems and re-push, potentially coming at some cost.
Solution
I needed to come up with a solution that handles all such tasks before my pushes to the remote repository (doesn’t matter if I forget or not) in an automated and generic manner. Obviously I needed a script that handles the tasks when run and a mechanism that allows me to run this script before every push
.
Pre-Push Hook
After a short research step I realized that Git Hooks
are just great tools that one can use for such purposes. There are a number of types of these hooks (you can see the full list from here) and pre-push
was the one that I was looking for to get rid of my problem. Using pre-push
hook one can control the complete flow of any push
operation for the codebase on underlying git repository.
For instance Running (any kind of) Tests could be a good starting point. Using that, one can ensure that no code that fails on tests is pushed to the remote repository. Another good example (especially for my case :) could be Regenerating all outdated mocks and with that, making sure that there exists no outdated auto-generated mocks on the remote. In addition to these two, we may also want to execute some other tasks for push convenience or, again, something that contributes to fail-free pipelines in the remote VCS.
The hook is just a bash script that is called by Git after a git push
command but before anything has been pushed. If this bash script exits with a non-zero
status git ensures nothing will be pushed. A sample pre-push
script could be as follows:
REMOTE="$1"
URL="$2"
read LOCAL_REF LOCAL_SHA REMOTE_REF REMOTE_SHAfunction run_tests() {
echo "Running all tests..."
go test ./go/... -p 1
if [ $? -eq 0 ]; then
echo "All tests run successfully 🎉🎉🎉"
else
echo "Tests failed ❌ ❌ ❌ aborting push!"
exit 1
fi
}function main() {
echo "Starting Custom pre-push hook..."
if [ -z "$REMOTE_SHA" ] && [ -z "$LOCAL_SHA"]; then
echo "No new push is on the way, skipping pre-push tasks..."
elif [ ! -z "$REMOTE_SHA" -a "$REMOTE_SHA" != " " ] && [ ! -z "$LOCAL_SHA" -a "$LOCAL_SHA" != " " ]; then
echo "New commits detected since last push, better run tests..."
run_tests
else
echo "Unknown pre-push hook usage ❌ please consult to the author of the hook with your use case..."
fi
echo "Successfully Executed pre-push hook, now pushing..."
}main
exit 0
Under your repository’s .git/hooks
folder you can find samples for each type of Git Hook with descriptions of the parameters and inputs passed by Git to the bash scripts.
My Custom Pre-Push Hook
To promote the ease of adding/ordering/removing pre-push tasks and to have the minimal maintenance cost possible of the hook in future, I have created a Task Queue
and aTask Template
structure in my custom Git Hook implementation in collaboration with the way that Git executes these hook scripts. Related parts of the hook is given below in bash syntax (some parts omitted for brevity).
/* ... */function run_all_tasks() {
/* ... */
i=0
TASKS=$1
for TASK in "${TASKS[@]}"; do
((i++))
$TASK $i
done /* ... */
}
function get_tasks() {
/* ... */ TASKS=(check_basename check_upstream)
BACKEND_TASKS=(auto_generate_mocks run_unit_tests run_integration_tests commit_backend_mock_changes) FRONTEND_TASKS=(run_cypress_tests)
/* ... */ if (git diff --stat $(git rev-parse --abbrev-ref --symbolic-full-name @{u}) | grep -q ".go"); then
## Changes detected for Backend, Adding Backend tasks ##
for BET in "${BACKEND_TASKS[@]}"; do
TASKS+=($BET)
done
fi
if (git diff --stat $(git rev-parse --abbrev-ref --symbolic-full-name @{u}) | grep -q ".ts\|.tsx\|.js\|.jsx\|.css\|.html"); then
## Changes detected for Frontend, Adding Frontend tasks ##
for FET in "${FRONTEND_TASKS[@]}"; do
TASKS+=($FET)
done
fi
fi
run_all_tasks $TASKS
}
function main() {
/* ... */ ## Run pre-push tasks only if a new push is on the way ##
if [ -z "$REMOTE_SHA" ] && [ -z "$LOCAL_SHA"]; then
## No new commits detected, skip pre-push tasks ## elif [ $LOCAL_SHA == $NEW_BRANCH_SHA ]; then
## Remote Branch Deletion, skip pre-push tasks ##
elif [ ! -z "$REMOTE_SHA" -a "$REMOTE_SHA" != " " ] && [ ! -z "$LOCAL_SHA" -a "$LOCAL_SHA" != " " ]; then
## New commits detected, run pre-push tasks ##
get_tasks else
## Invalid Usage, skip pre-push tasks ##
fi
}
main
In addition to this hook template, I’ve also uploaded a sample hook with following sample tasks (most of which are the ones that I’ve used to solve my actual problem mentioned at introduction:) :
check_basename
Checks whether working directory is set correctly by Git, Common Task.check_upstream
Checks whether currentHEAD
is behind upstream or not, for any push to go through, the localHEAD
should be ahead of the upstream branch, Common Task.auto_generate_mocks
Checks the files that has committed changes since last push, if detectsinterfaces
with an existing mock and has changes, regenerates the mock interfaces for them, Backend Task.run_unit_tests
andrun_integration_tests
are pretty self explanatory I believe :), Backend Tasks.commit_backend_mock_changes
Commits all the regenerated mock interfaces (if any), Backend Task.run_cypress_tests
Run tests written for Frontend withcypress
, Frontend Task.
If any of the above tasks fail, push will not succeed and an error will be thrown. Above tasks can be modified, enriched, appended or prepended with minimal effort using the general pattern I’ve used for existing tasks. Just implement a new function
for the new task and add it to the corresponding queue
with correct ordering.
For the full implementation check out this repository which contains a template sample form of the actual pre-push
hook I’ve implemented for our Git repositories at Picus.
How do I Enable it?
The default path for Git hooks is .git/hooks
and Git looks for special-named scripts in there, namely looks for a script named as pre-push
in our case. If such a script exist in there then git pre-push
hook is enabled and Git is going to run that script before each push.
I Can’t Share it Though
But as you can see this folder is under .git
so the hook will not be visible to the others if you work on the repository collaboratively. If you want to get your hook on the remote repository as well then you need to configure the core.hooksPath
of your local git repository as follows:
git config core.hooksPath hooks
Above command sets the hooks folder for Git to a folder named hooks
in the current directory, you can set to any path you wish. Now Git is going to look for hooks in this folder instead of .git/hooks
.
I’ve Enabled But How do I Bypass it?
If you don’t want your push to be reviewed by pre-push
hook then you can use:
git push --no-verify
This is going to push your new commits to remote directly.
Bonus: Pre-Commit 🎉
I loved using hooks. I found them very useful from certain aspects, hope you also find them useful and actually try them sometime. After my pre-push
implementation we have noticed that a tiny script that runs before every commit could also be very useful.
Therefore we have then implemented a pre-commit
hook that takes care of the pre-commit tasks such as formatting the code etc. I have uploaded a sample for pre-commit
too in my sample hooks repository I mentioned earlier.
Thanks for reading. Throughout this story I tried to describe my experience with git pre-push
hook, what my problem was and how did I solve them using the hook. I hope the story was clear, informative and, most importantly, fun :) Please feel free to share your opinions on comments section.