Using Githooks to DRY out GitHub Actions

A colleague recently asked me to summarize the “why” and “how” regarding our pre-commit hooks

Edit: Since this article was written, the feature landscape for GitHub has matured greatly, and the need for this workaround is less preferable compared to the templates offered by GitHub Actions. A good tutorial can be found here.

Octobi Wan Catnobi! (https://octodex.github.com/)

1. Wait, what?

For my day job, I contribute to an open source program designed to validate cloud infrastructure resources by probing for known weak points.

Our internal team manages its development infrastructure resources using a GitHub repository that’s entirely dedicated to executing Terraform through the GitHub Actions pipeline.

Between our deploy, destroy, and multiple validation CI workflows, we have several different YAML files that end up doing the same steps. The problem is that GitHub Actions doesn’t support YAML anchors, which are designed to reduce code repetition in situations such as this.

Using the standard GitHub Actions features, I need to either slam everything into one workflow, or copy+paste all common code (and remember to do it again for the slightest change to the common code later).

In the end, the risk of drift and the headache that comes with maintaining this type of work led me to build my own solution.

I decided to set up a pre-commit hook that will read a set of “anchored” files from one location, parse the files to be compatible with GitHub Actions, and then write the output to another file in the workflows directory.

The logic (linked below) will:

1. Read a file containing common logic for my workflows formatted into YAML anchors from .github/anchored (custom directory)
2. Read a directory of YAML files that harnesses the aforementioned anchors
3. Parse the common code into the appropriate locations and remove any evidence of the anchors
4. Write the parsed content to a new file into .github/workflows where it will be recognized by CI
5. Repeat for all files in .github/anchored

2. Why choose this solution?

I chose this for several reasons…

1. I didn’t want to cave-in and just repeat myself
2. I didn’t want to cave-in and just use a complex workflow for all situations
3. It’s not that hard, because Python’s ruamel parses anchors automatically
4. I know that anyone contributing to this repository will be competent enough to benefit from this approach if they choose to use it

3. How was this built?

1. I created a new directory and file.githooks/precommit (no extension) and then told my local git repo to use the new directory for all githooks by running the command git config core.hooksPath .githooks. Any other contributors can run the same command to use this githook as well.
2. I added a shebang to .githooks/precommit to tell it to run as Python, and then built the necessary parsing logic (Feel free to steal it, I just ask that you drop a star to let me know you liked it!)
   To explain one difficult design decision… I wasn’t able to get my parser to output YAML that was both human-readable and free of all anchor content. Because of this, my parser generates nearly-unreadable machine-generated code for the “functional” workflows.
3. I added a new directory called .github/anchored/ and moved all of my human-readable “anchored” workflows into it. I also created a new file called .github/anchors.yml which will be parsed into each of the anchored files by the precommit hook.

After the above was finished, every git commit... that I make locally will trigger the script.

The precommit will always quickly run through the workflows to see if anything would be changed by parsing. If they would be changed, it’ll git addthe changes to the current commit.

This adds about a second to every git commit... but that’s nominal compared to the time it saves.

Now, I never need to worry about drift between similar code checks- and if I make a change to one, that change will be applied to all.