Faster pull request checks for modular iOS app

Photo by NordWood Themes on Unsplash

This blog post is a follow-up of Enhancing XcodeGen for simpler maintenance of dependencies in modular iOS app . In the mentioned article I showed you how we enhanced XcodeGen format so only direct dependencies of modules are needed to be specified in project definitions. In this article I would like to present to you a solution that made our PR checks faster, as we run them only for modules that need it. It’s best if you first read the mentioned article to fully understand the issues that we faced. Anyway, I’ll start with a quick recap of the problem.

Initial problem and solution

In the sample setup we had 3 projects defined with XcodeGen:

• “ModularApp” application
• “FeatureModule” dynamic framework imported by the application
• “ApiModule” dynamic framework imported by the “Feature” module

The problem that we had was that in the YAML definition of the application project we needed to specify all the dependencies of “Feature” and “ApiModule” frameworks. Similar situation was with the unit tests target of the “Feature” framework — it needed to import all the dependencies of the “ApiModule” framework. The solution that I proposed was a simplified XcodeGen format — thanks to it, we needed only to specify the direct dependencies of every module. We have implemented a python script that analyses all the project definitions in the codebase, adds the indirect dependencies to them and produces a resolved version of the YAML files.

As we already traverse through the dependencies tree, as a side effect of the script, we save it and dump it to a JSON file. Thanks to it, we obtained a very handy representation of the dependencies between targets. It’s saved in the repository. You can also see how it looks below:

{ 
  "ApiModule": [], 
  "ApiModuleTests": [ 
    "ApiModule" 
  ], 
  "FeatureModule": [ 
    "ApiModule" 
  ], 
  "FeatureModuleTests": [ 
    "FeatureModule" 
  ], 
  "ModularApp": [ 
    "FeatureModule" 
  ], 
  "ModularAppTests": [] 
}

Scalable pull request checks

In our team we always longed for reducing time of our PR checks. As for complex iOS apps running all the unit tests may take a lot of time (like dozens of minutes), we wanted to run tests only for modules which could be affected by changes in the pull request.

Let’s say you modify only code in “ApiModule”. Then it totally makes sense to run only unit tests for “ApiModule” and for “FeatureModule” targets, as the “ApiModule” was affected itself and “FeatureModule” directly depends on the “ApiModule” by importing its public interface. You don’t need to run the unit tests of the “ModularApp” target, as it wasn’t directly affected.

As you have seen in my previous article, in my team we really like to improve our work performance using scripts. They proved to be helpful again when working on this challenge. But first let’s see what changes we made to our codebase setup.

Firstly, we needed to make sure that our module project directories exist on the main level of the codebase directory and that they are named the same as projects that are defined in the YAML files. Also each project should have exactly one target (not counting unit/ui test targets) with the same name as the project. This way we ensure that we properly identify changed targets.

Secondly, we created a “Tests” project located in the “Tests” directory. Its definition contains references to all our modules. It also defines the “UnitTests” scheme with test targets of all our modules:

name: Tests 
options: 
  bundleIdPrefix: com.fandom 
projectReferences: 
  ModularApp: 
    path: ../ModularApp/ModularApp.xcodeproj 
  FeatureModule:
    path: ../FeatureModule/FeatureModule.xcodeproj 
  ApiModule: 
    path: ../ApiModule/ApiModule.xcodeproj 
schemes: 
  UnitTests: 
    build: 
      targets: 
        ModularApp/ModularAppTests: [test, run]
        FeatureModule/FeatureModuleTests: [test, run] 
        ApiModule/ApiModuleTests: [test, run] 
    test: 
      targets: 
        - ModularApp/ModularAppTests 
        - FeatureModule/FeatureModuleTests 
        - ApiModule/ApiModuleTests

Thirdly, we defined a “Tests” workspace containing references to all our module projects, as well as “Tests” project. We’ll run the tests against it when we find out which unit tests need to be run.

When we had our setup ready, we created a python script that resolves which unit tests will be run. You can find it here .

It has few steps.

1. It takes a list of modified projects based on git diff command run against the base branch. It does it by getting first-level directory names of the modified files — that’s why it’s important that projects lie in directories named exactly the same.
2. We load the “Tests” project YAML definition. Then we iterate over test targets defined there and compare it with the list of modified projects.
   - We check if the project containing the test target belongs to a list of modified projects
   - We check if any of the project dynamic framework target’s dependencies belongs to a list of modified projects
   - We check if any of the test target’s dependencies belongs to a list of modified projects
3. If none of the above conditions are met, the test target is removed from the YAML definition of the “Tests” project.
4. The new YAML definition is saved as a project-resolved.yml file, which will be used to generate Tests.xcodeproj.
5. It calls XcodeGen to generate the Tests.xcodeproj

When we have our project generated, we just need to open the Tests.xcworkspace and run the “UnitTests” scheme.

Conclusions

This solution gives us measurable benefits if it comes to PR checks waiting time. Running all our unit tests currently takes about 15 minutes. With this solution, a PR check takes on average about 6.8 minutes. Of course there are still situations where we need to wait a lot of time when we modify a widely used module (it’s hard not to fall into the “Helpers” module trap imported all over your application). However, we are making steps to reduce the amount of code that needs to be put in such modules. Another way to improve the solution is to detect the changes made only to “public” interfaces of the modules and run unit tests on their dependencies only when such code is modified. However you need to consider how often you make non-public changes in an application consisting of multiple modules and whether the profit will be greater than the cost if introducing such a solution.

Originally published at https://dev.fandom.com.