Configuring Github Actions in a multi-directory repository structure
Hello World!
It’s nice to be able to write again and share small pieces of information that may save developers hours of debug time.
Today, I recount some of the challenges I faced in setting up a CI/CD via Github Actions for a recent project that I worked on specifically for a scenario where you have a multi-directory repository structure, mostly originating from cases where you have both the frontend and backend codes hosted on the same repository. If you don’t still quite get the picture, I have attached an image below for clarity. Now I’m sure you fully understand the scenario. Great then 😃 . I also proffered solutions and added link to resources that may aid further study. So if this sounds interesting to you , sit back and take a cup of coffee while we save you future hours in debug/research time.
Note:
1. Although the majority of the implementation here is biased towards setting up CI/CD via Github Actions for Flutter(Google’s Mobile Framework), however I have tried my best to communicate general ideas that can be transferred to any project you may be implementing CI/CD via Github Actions for.
2. This article assumes that you are familiar with or have previous experience setting up CI/CD via Github Actions and with Flutter(Google’s Mobile Framework) as this is in no way an introductory article for both concepts.
BIG PICTURE
The big picture which this article is based on is the initial problem which was encountered whilst trying to set up CI/CD via Github Actions for a multi-directory repository structure. I specifically wanted to set it up solely for a mobile project in the repo using Flutter(Google’s Mobile Framework). However while I began setting it up, I immediately became aware of some issues I would have going forward for which the solution was not immediately apparent. In the next sections, I give a summary of the initial issues I foresaw and the solutions I was able to discover after a bit of research and debugging.
THE PROBLEM
The initial concerns/issues I had while going about setting up CI/CD via Github Actions are summarised below:
- How could I set up my workflow file such that I am able to run my jobs specifically for the folder in the repository I am currently developing in (in this case the Mobile folder).
- After finding a solution to the problem above, how can I make sure that I only run specific jobs based on only file changes in that folder (in this case the Mobile folder). This would prevent a scenario where a push made by another developer working on another folder (i.e Backend) would not trigger my Github Actions to run which in this case would be an unnecessary run because the push did not contain any file changes from the Mobile folder.
THE SOLUTION
Armed with a proper understanding of my problem and with a firm belief in the power of open source to find solutions to these problems, I then proceeded to visit the developers favourite search engine to begin to find possible solutions.
For the first problem:
How could I set up my workflow file such that I am able to run my jobs specifically for the folder in the repository I am currently developing in (in this case the Mobile folder).
I quickly found an article by Anower Jahan Shofol (https://dev.to/shofol/run-your-github-actions-jobs-from-a-specific-directory-1i9e).
The general idea was that you could set different working directories for specific jobs that you had to run in your workflow. So for example if I had to run a specific job that was supposed to be Backend related, I could set the working directory for that job to be the Backend folder.
A snippet of this solution in code can be seen below as in our .github/workflows/master.yml file:
For the second problem:
After finding a solution to the problem above, how can I make sure that I only run specific jobs based on only file changes in that folder.
After a bit of search and trial and errors with some tools, I finally found an open source Github Action by Michal Dorner (https://github.com/dorny/paths-filter#examples). The main gist as stated in the project’s README file is that it is a GitHub Action that enables conditional execution of workflow steps and jobs, based on the files modified by pull request, on a feature branch, or by the recently pushed commits. After reading a bit through the documentation, I found out that I could add filters based on certain folders I wanted to watch for file changes and then conditionally decide to run parts of my workflow based on if files in the folders i’m watching has been changed.
A snippet of this solution in code can be seen below as in our .github/workflows/master.yml file:
IMPLEMENTATION
Based on the solutions explained to the problems itemised above, you can have a general workflow file that looks as below which would work for you if you happen to find yourself in the same scenario I did previously.
With this we have come to the end of a wonderful ride. I really hope this article was beneficial to you and saved you some research/debug time. If you found this article useful, please leave a clap, comment or share with anyone it may help. If you find any thing incorrect or lacking in explanation, please leave a comment and I will edit accordingly.
Thanks and keep building!