Git Pre-Commit — Part 2 — The Framework

Today’s article mainly focuses on the pre-commit open-source framework for managing and maintaining multi-language pre-commit hooks.

Please refer to our first discussion over Git Pre-Commit — Part 1 — Zero Trust My Code! for a quick overview.

The advantages of the Pre-Commit Framework are the community ‘behind-the-lines’, performance (if properly cached), and that one can use it for repositories where multiple languages are needed simultaneously (Python, Terraform, JavaScript, etc.)

All we want to do is to make sure that before committing anything, one runs the Code-Formatter/Analysis on Source Code to comply with agreed-on practices. Of course, we can manually run a Code-Formatter every now and then, but the point is:

“Why not just make it automatic?”

How to configure it?

In a nutshell, zero configuration setup — nothing is needed beyond the .pre-commit-config.yaml file!

The steps are detailed below:

Let’s assume Python & Git are already installed on our machine.

1. Get the Pre-Commit framework packages installed first.

For detailed setup instructions, follow the link.

For Linux-based machines, just type:

pip install pre-commit

2. Create the `.pre-commit-config.yaml` Configuration File and modify its content as below:

At the project’s root, we create a new file called .pre-commit-config.yaml.

Within the created .pre-commit-config.yaml file, we start by defining the structure where we specify which hooks to implement and which checks to trigger. As an example:

repos:
- repo: https://github.com/pre-commit/mirrors-prettier
  rev: "v2.7.1"
  hooks:
     - id: prettier

Above, we used “Prettier.io,” a code-formatting tool, to enforce a consistent style across all files within the repository.

3. Setup the Git Hooks

Use `pre-commit` to install the Git Hooks into our Git Hooks folder (This is a one-time operation):

pre-commit install

#Run a first check of the pre-commit hook
pre-commit run -a

For advanced pre-commit capabilities, please check the advanced features article.

4. Now, we can make a small change and try to commit our code.

# Please remove the manually created file from .git/hooks/pre-commit
# This file is created in the above part
rm -rf .git/hooks/pre-commit

# Commit your code
git commit -am "Made some changes"

By triggering a commit, we expect an output of the pre-commit execution.

Pre-commit hooks triggering Prettier.io

It is not really required to do a commit every now and then to have it tested;

What we can do, though, is to run the below check on all local files whether they changed or not)

# Run pre-commit on all files
pre-commit run --all-files

# Or Short version (-a is short for --all-files option)
pre-commit run -a

We can simply run one hook at a time by specifying the Hook name as below:

# Run one hook at a time
# pre-commit run -a <HOOK-ID>
pre-commit run -a check-yaml

We can skip the checks by appending the -n option when committing code:

git commit -m "Made some changes" -n

Pre-Commit — Plugins

Let’s add some other checks to our previous .pre-commit-config.yaml configuration. The file should be like the below config:

repos:
- repo: https://github.com/pre-commit/mirrors-prettier
  rev: v2.7.1
  hooks:
    - id: prettier
- repo: https://github.com/pre-commit/pre-commit-hooks
  rev: v4.3.0
  hooks:
    # Verify YAML syntax
    - id: check-yaml
      name: "Check YAML"
      args:
        - --allow-multiple-documents
    # Protect Master from direct checkins
    - id: no-commit-to-branch
      name: "No Commits to Master"
      args:
        - --branch=master
- repo: https://github.com/bridgecrewio/checkov.git
  rev: 2.1.121
  hooks:
    - id: checkov
      name: "Checkov"

The second repo, shown above, points to the BridgeCrewIO Checkov project and will trigger a hook for Checkov checks on Terraform files.

We first need to ensure to keep track of the .pre-commit-config file using git add .

Here, our checks failed for several reasons: We kept working on the master branch, which we prohibited using our hooks.

This shows that `Prettier` already applied some formatting to a file tracked by Git.

To resolve the “No Commits to Master” issues, checkout to feature/or developbranch, and append the last updates made by Prettier.io to the files, as shown below.

Checkov has no files to check, so the test is skipped. Voluntarily add a basic Kubernetes Manifest Yaml file (Pod creation as an example), commit the code, and check Checkov execution for non-compliant files.

Pre-Commit — Caching Mechanisms

We can’t underestimate the value of caching and how it can benefit recurrent pipeline executions. Performance is a crucial KPI here, particularly if conducting heavy checks.

The pre-commit framework caches its execution to PRE_COMMIT_HOMEwhich basically is ~/.cache/pre-commit folder.

To implement Caching for the Pre-Commit Framework, one can use the below example. Unfortunately, this is not a universal solution and is just valid for Gitlab-CI.

pre-commit-job:
  stage: linting-stage
  image: <XYZ>
  # Run the job only when the file `.pre-commit-config.yaml` exist
  rules:
  - exists:
      - .pre-commit-config.yaml
  # Enable Caching for better pre-commit performance
  cache:
    key: pre-commit
    paths:
      - $CI_PROJECT_DIR/.cache
  variables:
    PRE_COMMIT_HOME: $CI_PROJECT_DIR/.cache/pre-commit
  script:
    # Always upgrade pre-commit/checkov to latest released version
    - pip install --upgrade pre-commit
    - pip install --upgrade checkov
    # Trigger pre-commit checks on changed files only (Perf.)
    - pre-commit run --files $(git diff-tree --no-commit-id --name-only -m -r "$CI_COMMIT_SHA")

Feel free to check the implementation for other CI pipelines following this link.

Pre-Commit — Team collaboration!

Git Hooks are not checked in Source Control, which makes it difficult to share and enforce within a team.

Policy enforcement hooks should be server-side [According to the Git Docs].

That being said, it may become thorny to manage when teams try to work with Hooks.

One of the solutions is that one can set up a skeleton for Git using the init.templateDir option, so when the repo is cloned, the engineer gets the hooks automatically set up for him. For additional implementation details, I invite you to check the article “Automatically enabling pre-commit on repositories.”

It’s recommended to enforce the pre-commit run --all-files in your CI-pipelines as this is your last line of defense before making changes on the target environment.

To Conclude

We should recognize that making the hooks fast & easy to assimilate is much more useful and less impactful to productivity.

To finish, I recommend reading an interesting StackOverflow article: “How to use pre-commit to correct commits and merge requests with GitLab-CI automatically.”

Thank you so much for reading me :)

Please do not hesitate to share your feedback or additional input.