GSoC 2022 with CNCF — Final Report
Hello people 👋🏻
During the past 3 months I have been working with Cloud Native Buildpacks under the org CNCF on the project to standardise cache options used in the the pack CLI tool
What is pack?
In the basic context of the user being an app developer, pack makes it super easy to convert your source code to OCI images that run on any cloud (AWS, GCP, Azure, Heroku, etc.)
Here’s a ruby app source code being converted to a runnable docker image
What did I do?
Pack relies on a couple of caching mechanisms internally to facilitate faster build times, in ephemeral environments (CI/CD pipelines). These cache options were not configurable by the end user, I worked on this limitation to standardise a consistent interface for all the different cache options and expose those as configurable flags.
Have a look at the detailed explanation in the 📃 project proposal.
Work Done
Pack has two cache types namely,
- Build Cache — Cache that contains the layers that were created by buildpacks, that were marked to be cached, for example this could be a simple dependencies folder
- Launch Cache — Cache that contains the layers that were marked to be launch layers, these, launch layers are used internally to speed up subsequent build processes
Now both these cache types could be saved in the following formats
- Image — Normal OCI images that may be on the host system or on a remote registry
- Volume — Named volume bind managed by docker
- Bind — Directory binds from the host machine to the pack builder
The idea was to introduce a new flag --cache that would take in variadic arguments for example
pack build --cache 'type=<cache-type>;format=<cache-format>;name=<cache_name>'
# cache-type: build, launch
# cache-format: image, volume, bind
# name: cache name or location of cache folderThe whole project revolved around supporting these different combinations for cache
1. Support for image cache
Image cache were previously supported using the --cache-image flag that would create an image on a remote registry, this was added to the new flags, while also retaining the old flags
pack build —cache ‘type=build;format=image;name=my-cache’
buildpacks/pack@6504e8731167ae94f5cb477680bdee3a6a87959b
buildpacks/pack@7d267bcc6911cdefea08f3c1bb799d2fb6b850fd
buildpacks/pack@c1deb54451bde9ac4b338785b5a61a8cf9f92f23
2. Support for volume cache
Volume cache were not configurable, so support for specifying volume cache format for both launch and build cache along with configuring names was added
pack build —cache ‘type=build;format=volume;name=my-cache-volume’
buildpacks/pack@397c959c21e804a46e18e948fedd6009ed56df5e
buildpacks/pack@1eb52c0bc03cb327f1c2e62b55c23dba33d92776
3. Support for bind cache
Bind cache type was not supported by pack at all, in some restricted environments this type of cache is necessary to have access to the host filesystem, full support for this cache type was added
pack build —cache ‘type=build;format=bind;name=./bind-folder’
buildpacks/pack@56d8dc09538f4f17d4856a82ab7a2eeac9b2388d
4. Miscellaneous
There were a couple more things achieved both before and during the GSOC pe
- buildpacks/pack#1444 Support for embedding git metadata like tags, commit hash, release info in the final build image
- buildpacks/pack#1437 Error handling issue
- buildpacks/pack#1427 Update the
pack inspectcommand output to include project’s current working directory - buildpacks/pack#1439 Update file filter logic influencing directory include and exclude logic
Closing thoughts
Overall my GSoC journey has been full of learnings, I exploring the pack and lifecycle codebase earlier this year knowing nothing about golang or CLI tools in general, after all the work that was put into reading documentation, understanding codebase, along with the whole process from a project’s source code to a standalone image through various lifecycle phases I am pretty confident in my ability to work with both GO and CLI tools.
🙏 I want to thank the whole buildpacks community, especially my mentor jromero who has been super supportive throughout the journey.
🉐 If there was a piece of advise I’d give to someone who wants to get into open source software then I’d suggest them to
1. explore the project codebase
2. if things are not clear then read the documentation and get back to exploration
3. if things are still not clear then search for issues, there’s a high chance your doubts are already asked by someone in the past
4. if there’s still issues then compile a list of doubts and connect with a community member over a call to discuss the details in a go
Contributing to open source will definitely help you grow as a developer and it changes your whole perspective of writing code while also caring about code quality, maintainability and testability, you will learn and experience real world software development practices in a way that no other method can teach you.
