Improve your CI-CD-Workflow with Git-Notes
As we all know, when working on a project, there is a wealth of information and documentation that needs to be managed effectively. However, more often than not, this information is scattered across various tools and systems, making it difficult to locate and access when needed.
For example: In most modern projects CI/CD-Pipelines are used to automatically build and test our applications. While building the application many reports are generated, that need to be stored along the code: SBOMs, Test Reports, QA approvals. Most of this stuff ends on various places. Build logs are only available in the logout put of the pipeline run, which is often not archived. Reports from third party tools like SonarQube are stored in the system self. But these systems are not made for long term archiving and the connection to the other reports is given.
Manual Tests reports are often Word or Excel sheets stored in some central wiki system.
This means that crucial information about your software development process is spread out across different software systems, making it challenging to consolidate and retrieve later on.
Why git-notes is a good idea?
This is where git-notes comes in as a solution. With git-notes, you have the opportunity to store all the pertinent information directly within your git repository, seamlessly linked to the relevant git commit. This has significant advantages in terms of organization and accessibility.
By utilizing git-notes, you can associate important details, such as testing reports, directly with the corresponding commits in your repository. This integration enables you to easily trace and retrieve relevant information whenever you need it. No more wasting time searching through disparate systems or struggling to find the right document buried deep within a network drive.
Furthermore, storing this information within your git repository ensures that it remains version-controlled, just like your source code. This means that you can track the evolution of the associated information over time, maintaining a comprehensive history of your project’s development process.
Additionally, by centralizing all this information within the repository, it becomes easier for team members to collaborate and share knowledge. Everyone can access and contribute to the shared pool of information, enhancing communication and fostering a more streamlined and efficient workflow.
What is git-notes?
With git notes it is possible to add information to an already existing commit without modifying it or adding a commit to your development history. You could add any kind of information to a commit, but in this blog post, we use the mentioned results of your build pipeline.
Let’s have look at a simple example. We take a normal commit that for example introduces a new feature.
$ git log
commit c81913310eb923b5f2de5cbb8cfc72394493c0fa
Author: Max Mustermann <max.mustermann@example.com>
Date: Thu Oct 13 13:18:37 2022 +0200Add new feature
We tested this feature, and we want to add an approval note. We could do this with the git notes add command:
$ git pull <some-feature-branch>[..] build$ git notes add -m 'Tested-by: Michael Graf' <commit-hash>
Afterwards we could see our note by git notes show.
$ git notes show
Tested-by: Michael Grafgit notes are also displayed in git log.
$ git log
commit c81913310eb923b5f2de5cbb8cfc72394493c0fa
Author: Max Mustermann <max.mustermann@example.com>
Date: Thu Oct 13 13:18:37 2022 +0200Add new featureNotes:
Tested-by: Michael Graf
Which is a cool thing.
Multiple notes with git notes
Internally git notes are just blobs that are stored on a special branch under refs/notes/commit. On this special branch each file corresponds to a git object (commit). As there is now a file for our commit git notes add only works once.
$ git notes add -m 'Tested-by: Désirée Graf'
error: Cannot add notes. Found existing notes for object c81913310eb923b5f2de5cbb8cfc72394493c0fa. Use '-f' to overwrite existing notesIf someone else wants to approve our feature, she needs to use git notes append
$ git notes append -m 'Tested-by: Désirée Graf'$ git log
commit c81913310eb923b5f2de5cbb8cfc72394493c0fa
Author: Max Mustermann <max.mustermann@example.com>
Date: Thu Oct 13 13:18:37 2022 +0200Add new featureNotes:
Tested-by: Michael Graf
Tested-by: Désirée Graf
While this is working for a manual QA process, storing JUnit test reports and other results in the same text file will be a mess. But we don’t have to store everything in one single text file. As mentioned before, git notes use refs/notes/commit to store notes. But with the --ref parameter we can change that. This gives us the possibility to store different notes in differente refs.
For example, we could store our Junit report under refs/notes/junit and our SonarQube findings under refs/notes/sonarqube. To directly upload a text file as a note we can use the-F parameter. And like with normal branches under refs/heads/, we don’t need to write the full ref like. Git will automatically prefix our refs with refs/notes if needed.
$ git notes --ref junit add -F ./build/test-reports/junit.xml
$ git notes --ref junit show
...Binary notes with git notes
Sadly, not all reports and notes are in text format. Some are binary and-F only supports text files. To use git notes with binary notes we need a “workaround”. We must use git hash-object to store the binary note before as a blob and then reference it as a note. This sound complicated but it isn’t. We could do this in a single bash line:
$ git notes --ref=pdfreports add -C $(git hash-object -w report.pdf)To show the binary report we just need git notes show.
$ git notes --ref=pdfreports show > report.pdfSynchronize git notes between repositories
Sadly, notes are not pushed and fetched out of the box. We must explicitly push and fetch them.
$ git fetch origin "refs/notes/*:refs/notes/*"
$ git push origin "refs/notes/*"Example usage in a pipeline
With all the commands above we could easily extend our CI-Pipelines to store the results of our build by using git notes. The following azure pipeline definition is just an example how it could look like. The interesting part starts at line 45.
The first step after the build and the report generation is to setup git (line 45–46). The next lines are storing the generated reports with git notes and line 51 appends the note under the standard ref (refs/notes/commit) that the build was successfully.
After that, all notes are pushed into the git repository (line 52).
As you can see, with git notes it's possible to easily store all kinds of logs, test results and reports in your git repository, without the need to commit this information or rewrite the history.
The down side of git-notes
While git-notes offers several benefits, it’s important to consider its limitations and the challenges that come with its adoption.
Firstly, it’s worth noting that git notes is not a new feature. It has been a part of Git for more than a decade. However, despite its potential, it has not gained widespread adoption or recognition among developers. This lack of awareness may be one reason why many individuals and teams have not fully explored or utilized this feature.
One of the contributing factors to the underutilization of git notes is its quirky nature and relatively complex usage. The process of adding and managing notes using git notes is not as straightforward or intuitive as some other Git commands and features. This complexity may deter users from taking full advantage of its capabilities.
Furthermore, due to the limited popularity and adoption of git notes, many user interfaces (UIs) and tools developed for Git have dropped support for this feature. This is unfortunate because git notes has the potential to significantly improve modern development processes. The absence of UI integration and tool support means that developers have to rely on command-line interfaces or custom scripts to work with git notes effectively, adding an extra layer of complexity and potential barriers to adoption.
Another drawback of git notes is the potential for increased repository size. When notes are added to the repository, they contribute to the overall size of the repository, especially when dealing with a large number of commits and extensive notes. This can impact storage requirements and make cloning and transferring repositories more time-consuming and resource-intensive.
In conclusion, git notes presents an excellent opportunity to enhance your software development process. By storing relevant information directly within your git repository and linking it to the appropriate commits, you can simplify organization, improve accessibility, and maintain a comprehensive historical record. This enables your team to easily trace and retrieve important details, such as testing reports and logs, whenever they are needed, fostering a more efficient and streamlined workflow.
However, it’s important to consider the downsides of git notes as well. Despite its potential benefits, git notes has faced challenges in terms of limited awareness, complex usage, and a lack of support from user interfaces and tools. The quirky nature of git notes and the additional repository size it can introduce may also deter some developers from fully embracing this feature.
Nonetheless, for projects that require comprehensive documentation, detailed logs, or extensive test results, git notes can still serve as a valuable tool within the Git ecosystem. It offers a centralized approach to information management, ensuring that crucial project details are version-controlled and easily accessible to team members.
Ultimately, the decision to adopt git notes depends on your team’s specific needs and preferences. Exploring its capabilities and understanding its intricacies can potentially lead to improved collaboration, enhanced knowledge sharing, and the delivery of high-quality software. By weighing the advantages and downsides, you can make an informed choice about integrating git notes into your software development process.
Thanks for reading! What are your thoughts about git notes? Do you know other cool lesser-known git commands?
