Reduce Turnaround Time for Pull Requests Over Multiple Module Projects
Don’t wait for dependent ‘pull requests’ in module projects to get merged.
It is getting more common to work on multiple module projects due to the benefits of modularization. Because of that, sometimes, we need to modify ‘:app’ project as well as ‘:module’ project at the same time so two pull requests need to be created, one in ‘:app’ project, the other in ‘:module’ project. In order to trigger the build for the branch in ‘:app’ project in this case, we are waiting for the pull request in ‘:module’ project to get merged because the change in the ‘:module‘ project should be applied to the build for ‘:app’ project. This workflow in serial order usually slows down the speed of review and approval process. I would like to share my experience to reduce the turnaround time for the workflow over multiple module projects.
What Makes Work Process Slow Down In General CICD Workflow
The review of a pull request requires the build which provides the reviewers with a couple of data points, such as compile error, tests result, code coverage, as well as code quality. CI server provides the build, and it is usually configured to build binaries, run tests, publish artifacts, and integrate other systems for code quality. In this article, it is assumed that there are separate build plans for pull request branch and master branch in each project, and the detail configuration of build plan is described here. In this case, the build for ‘app-pullRequest’ branch in ‘:app’ project needs to refer to the artifact from the build for ‘module-pullRequest’ branch in ‘:module’ project.
The general workflow for the two pull requests is as follows.
- create ‘app-pullRequest’ branch in ‘:app’ project and ‘module-pullRequest’ branch in ‘:module’ project.
- prepare the review for the ‘module-pullRequest’ branch based on the build for ‘module-pullRequest’ by build plan in CI server.
- proceed the review for ‘module-pullRequest’ and get approvals for the merge (the detail workflow for review process is not described here).
- get ‘module-pullRequest’ branch merged and publish the artifact of ‘module.aar’.
- prepare the review for ‘app-pullRequest’ branch based on the build which refers to the ‘module.aar’.
- submit the review for ‘‘app-pullRequest’ branch.
Preparing the review for ‘app-pullRequest’ should be deferred until the new module artifact gets ready during Step#2~Step#4 in workflow above. This shows the workflow for ‘module-pullRequest’ branch impacts that of ‘app-pullRequest’ branch and introduces longer turnaround time for the work process. If any problem of ‘module-pullRequest’ branch is figured out and it needs to be changed during the review of ‘app-pullRequest’ branch, we need to return to Step#2, and it will take longer time to get the review of ‘app-pullRequest’ branch ready.
Besides longer turnaround time, the reviewers are not able to review both pull requests at the same time because ‘app-pullRequest’ can’t be submitted until ‘module-pullRequest’ gets merged.
Make Turnaround Time Shorter and Extend To Multiple Module Projects
As described above, if both of ‘app-pullRequest’ and ‘module-pullRequest’ are involved, there are a couple of factors which make the work process slowdown in general workflow.
The detail workflow is as follows.
- create ‘app-pullRequest’ branch in ‘:app’ project and ‘module-pullRequest’ branch in ‘:module’ project.
- prepare the review for the ‘module-pullRequest’ branch based on the build for ‘module-pullRequest’ by build plan in CI server.
- publish the artifact of ‘module-pullRequest.aar’ to artifactory through the build plan for ‘module-pullRequest’.
- prepare the review for ‘app-pullRequest’ branch based on the build which refers to the artifact of ‘module-pullRequest.aar’ which is available by Step #3.
- submit the review for ‘module-pullRequest’ branch and get approvals
- submit the review for ‘app-pullRequest’ branch alongside with Step #5 and get approvals.
The Step #3 makes the artifact of ‘module-pullRequest’ available in artifactory, so that Step #4 to prepare the review for ‘app-pullRequest’ branch is able to start. This would remove the turnaround time to merge ‘module-pullRequest’ for the build of ‘app-pullRequest’ branch. Additionally, both of the pull requests would be able to be submitted at the same time because the build of ‘app-pullRequest’ branch doesn’t have to wait for ‘module-pullRequest’ branch to get merged any more. Additionally, it gives the benefit for the reviewers to be able to review all relevant pull requests at the same time with better understanding and context across projects.
Once the reviews of pull requests are done, we should get ‘module-pullRequest’ merged before merging ‘app-pullRequest’, otherwise, the build of target branch for ‘app-pullRequest’ is going to fail when merging ‘app-pullRequest’ because it doesn’t include the relevant change in ‘module-pullRequest’ branch.
For the sake of simplicity, it is assumed that the case has only one module project. The new workflow can be extended to projects which have more than one module project.
Summary
Usually, it takes more than one day from preparing a pull request to merging it after getting sufficient approvals from the reviewers. If ‘:app’ project includes one ‘:module’ project, we can save the turnaround time for ‘module-pullRequest’ to get merged before preparing ‘app-pullRequest’, at least more than one day. If ‘:app’ project includes multiple concatenated ‘:module’ projects, we can save the turnaround time in proportion to the number of concatenated ‘:module’ projects. In addition to saving the turnaround time, it provides the reviewers with better context across projects by all relevant pull requests at the same time.
Configuration Tips
This section shows the detailed configuration tips for the new workflow.
Use same branch name for ‘:app-pullRequest’ and ‘:module-pullRequest’ branches
The build of ‘:app-pullRequest’ needs to determine the dependency name for the artifact of ‘:module-pullRequest’. One of the recommendations is to use the same branch name for all relevant branches. It will give an easy way to specify the dependency name of ‘:module-pullRequest’ across projects.
For instance, the same branch ‘task/improvement-workflow-example’ needs to be created in ‘:app’ and ‘module’ projects.
Separate build configuration for pull request from production build
The build of ‘:app-pullRequest’ needs to apply different gradle build configurations for pull request other than debug or release build on master branch. We can define a new property, ‘pullrequestBuild’, to specify the build of pull request in build.gradle, and it is set by command option ‘-PpullRequest=true’ in CI server.
if(!project.hasProperty(“pullRequestBuild”)){
ext.pullRequestBuild = false
}Publish the artifact for ‘module-pullRequest’
Publishing ‘:module-pullRequest’ requires a unique artifact name to specify the artifact for it. The following example adds ‘-PR’ string to artifact ID and ‘-PullRequest’ string to artifact version to specify the build for the pull request. For example of ‘task/improvement-workflow-example’ branch name, artifactID is set to ‘module-PR’, and artifact version is set to ‘version-PullRequest-task-improvement-workflow-example’ for the artifact.
def moduleArtifactId = "module"
def moduleArtifactVersion = version
if(pullRequestBuild){
moduleArtifactId = moduleArtifactId + "-PR"
moduleArtifactVersion = moduleArtifactVersion + "-PullRequest"+"-"+ branch.replace("/","-")
}Determine the artifact for module depending on the existence of the artifact for ‘module-pullRequest’
Sometimes, the ‘module-pullRequest’ is not required depending on the scope of changes, and this should be considered when determining the name of the module artifact. One of the recommendations is to check the existence of an artifact for ‘module-pullRequest’ in artifactory first. If it exists, it refers to it, otherwise, it refers to the artifact from the master branch. The following is a ‘build.gradle’ configuration to check the existence of the artifact for ‘module-pullRequest’.
///////////////////////////////////////
// Evaluate the URL for the artifact //
///////////////////////////////////////
def checkModulePullRequestArtifact(String artifactoryUrl, String version) {
def artifactUrl = artifactoryUrl + "/com/sample/android/" + artifactPath + "/"
version = version + "-PullRequest" + "-" + branch.replace("/", "-")
artifactUrl = artifactUrl + version + "/" + artifactPath + "-" + version + ".aar"
return checkArtifactExist(artifactUrl)
}//////////////////////////////////////////////////////////////////
// Check the existence of the artifact for 'module-pullRequest' //
//////////////////////////////////////////////////////////////////
def checkArtifactExist(String artifactUrl){
println "artifactUrl: ${artifactUrl}"
try {
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection con =
(HttpURLConnection) new URL(artifactUrl).openConnection();
con.setRequestMethod("HEAD");
return (con.getResponseCode() == HttpURLConnection.HTTP_OK)
}
catch (Exception e) {
return false;
}
}
