iOS Continuous Integration with GitLab CI, Fastlane & OTA Installation

If you have ever worked on a project together with other people, perhaps you’ve had a situation in which you pulled some changes from a git repository and it turned out that something stopped working or the project stopped compiling. Continuous integration helps to avoid such problems. Automatic builds and tests execution after each commit are also useful when you are the only person working on a given project. With continuous integration, you can notice problems faster and fix them as soon as they appear.

So in this article, we’ll talk about:

• Why Gitlab?
• Gitlab CI and Runners
• Gitlab Runner installation
• Introduction to Fastlane
• CocoaPods, Bundler and Ruby gems permissions
• Configuring Fastlane
• Configuring Gitlab CI for our TestProject
• Over-The-Air IPA Installation from Gitlab
• Conclusion

Why Gitlab?

As perhaps you already know, Gitlab.com offers free git accounts also for private projects. Moreover, at the same time, it offers the option to configure an integrated CI. We can have everything in one place which is without a doubt a huge advantage. Furthermore, we also get Gitlab Pages, which is a feature that allows us to host a static website. We can use that for creating a solution for Over-The-Air IPA installation straight from Gitlab without using other services. Before we begin let’s assume that we already have a simple private iOS project named TestProject hosted on Gitlab.com and that we want to configure Gitlab CI for this project.

Gitlab CI and Runners

Gitlab.com offers its own shared servers that can be used to build private projects. However, due to the fact that none of them is running macOS, these servers can not be used to build applications for iOS. Therefore, in our case, we need our own build agent. To build iOS projects using Gitlab CI you need to connect your computer that should act as a build agent with Gitlab using an application called Gitlab Runner. This application installed on your computer will communicate with Gitlab service and if it receives information that it should build a project, then it will download this project, build it and send the result to Gitlab.com.

Gitlab Runner installation

To install Gitlab Runner on a computer that should be a build agent, first open the terminal and download Gitlab Runner using the following command:

sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64

Sometimes it may happen that you do not have the /usr/local/bin directory yet and you’ll see an error. In such case you just need to create this directory with the following command:

sudo mkdir /usr/local/bin

Then you can repeat the command that downloads the Runner. When the download is complete you must give permissions to execute the runner with this command:

sudo chmod +x /usr/local/bin/gitlab-runner

After that you can check if everything works simply by running this command:

gitlab-runner

You should see the help of the gitlab-runner program with a list of available actions. You can now connect your Runner with your project at Gitlab.com. On the project page, select the Settings tab and then the CI / CD option. Next to the Runners section, there is the “Expand” button so click it. There are two subsections. In the “Shared Runners” section, click on “Disable shared Runners for this project” because as I’ve already mentioned iOS projects can not be built using shared runners. In the “Specific Runners” section, you’ll see a registration token, which you will need in a moment. Now you can return to the terminal and execute the command:

gitlab-runner register

In the next steps, you have to enter the configuration for this runner. In our case, you can enter it according to this example:

Please enter the gitlab-ci coordinator URL:
  https://gitlab.com/
Please enter the gitlab-ci token for this runner:
  (the token found on Gitlab website)
Please enter the gitlab-ci description for this runner:
  [hostname] (press enter to leave the default value)
Please enter the gitlab-ci tags for this runner (comma separated):
  ios
Please enter the executor:
  shell

At this point, after refreshing the Gitlab.com website in the “Specific Runners” section you should see your new Runner. It is now configured but you still have to turn it on. You can do this in two ways, either start it as a service running in the background or simply run it in the terminal (using the gitlab-runner run command). The second option can sometimes be useful, for example, when something does not work and you want to see live what exactly gitlab-runner is doing. However, I recommend using the first option and install the Runner as a service that runs in the background so that you no longer have to start it manually. To do this, install and run the service using two commands:

gitlab-runner install
gitlab-runner start

At this point, on the Gitlab.com website in the Specific Runners section, you should see a green dot next to your Runner which confirms that it is working properly.

Introduction to Fastlane

To use Gitlab CI, we need a way to build and archive our project from the command line. Although we can do this without additional tools using the xcodebuild command, the better solution will be to use the Fastlane tool. If you have not used Fastlane before, in short, it is a tool which based on a simple script prepared by us can automatically perform various operations related to the entire application development process. For example: building a project, running tests, generating code coverage reports, sending builds to TestFlight, HockeyApp, etc., automatically generating screenshots from the application, sending build notifications and many other operations. Before we get into the details, it’s worth saying that Fastlane, similarly to popular dependency manager CocoaPods, is written in Ruby.

CocoaPods, Bundler and Ruby gems permissions

If you had the opportunity to use CocoaPods then most likely you encountered a situation in which different members of your team had installed different versions of CocoaPods on their computers. This could cause some problems. In extreme cases, building a project could work completely differently in different versions of CocoaPods. To make sure that our project is built on each computer exactly in the same way it is best to use everywhere exactly the same version of Fastlane, CocoaPods, etc.

To make sure that we use the same versions of all needed tools on each computer, we can use Bundler tool. One can say that Bundler is something similar to Cocoapods, but instead of a Podfile we have a Gemfile and instead of managing versions of iOS libraries it is used to manage versions of Ruby gems, for example, CocoaPods and Fastlane. Therefore, it is another tool which in some cases makes life easier and helps to avoid problems with incompatibility between particular versions of Ruby gems. The problem with different versions of CocoaPods is obviously related to the situation in which Pods are not kept on git repository. If the entire Pods directory is kept on git repository and you do not have to download external libraries to compile your project, then described problem does not occur and perhaps you won’t need Bundler in such case. Depending on your preferences and needs, you can decide whether it’s worth using Bundler in your case or maybe it is worth keeping Pods on git. In this text, I will focus mainly on Gitlab CI itself. I am talking about this issue, however, in order to bring some solutions to potential problems that you may encounter especially when your CI will be used not only by you but also by other developers.

In case you decide to use Bundler, it’s also worth mentioning potential problems with permissions when installing Ruby gems. In macOS, Ruby is already installed by default, but installing gems requires using sudo. Depending on which computer your CI will work on and who will use it, it may be a problem or not. If you want to use Bundler and you want everyone to be able to decide which gems are used on your CI and install them via the Gemfile, then Bundler must be able to install gems without sudo. The first and the most common solution to this problem is installing Ruby with RVM (Ruby Version Manager) as described here and using this installation instead of the system Ruby. Then all gems will be installed in the directory associated with this installation instead of the system directory. Another option is to configure the system Ruby to install gems somewhere else, for example in the home directory using the GEM_HOME environment variable as described here. I leave you with the decision whether you need to use this solution or not.

Configuring Fastlane

Let’s setup Fastlane so that we can build and export to IPA our test project from the command line. If you have not installed Fastlane on your computer, then according to the description on https://docs.fastlane.tools/getting-started/ios/setup/ you can do it with this command:

gem install fastlane -NV

As I mentioned before, this command may require sudo if you do not use RVM nor changed the gems installation path by setting the GEM_HOME environment variable. When Fastlane installation is finished, you can go to the directory with your project in the terminal and initialize Fastlane with the following command:

fastlane init

After that, a new fastlane directory will be created with a file named Fastfile. Alternatively, instead of fastlane init, you could simply create that directory and file manually. Open that file in any text editor and change its contents to the following:

default_platform(:ios)
platform :ios do
  desc "Build application"
  lane :build do
    gym(
      scheme: "TestProject",
      clean: true,
      output_directory: "build",
      export_options: {
        method: "development"
      }
    )
  end
end

Thanks to the above simple script, from now on it is possible to build an IPA file from the terminal using this command:

fastlane build

You can check if everything works properly by running the build from the terminal. If everything is ok then the IPA file will be created in the build directory according to what you can see in the above script. I set the export method to development so that I do not have to worry about provisioning profile now. Depending on your own needs, you can use other export methods, e.g. ad-hoc or enterprise. Fastlane scripts is a bigger topic for a separate article, so if you are interested in other operations you can perform in the Fastfile file, then I advise you to check the documentation here. In this text, I will focus primarily on combining this with Gitlab CI.

Configuring Gitlab CI for our TestProject

We have already configured Gitlab Runner and we are able to build our project from the terminal level. It’s time to make our project automatically build after each change on git. We need a configuration file .gitlab-ci.yml which we must create in the main directory of our repository. Then we should put in it the following content:

before_script:
  - export LC_ALL=en_US.UTF-8
  - export LANG=en_US.UTF-8
stages:
  - build
build_job:
  stage: build
  tags:
    - ios
  script:
    - fastlane build
  artifacts:
    expire_in: 90 days
    paths:
    - build
  only:
    - master
    - develop
    - feature/*

I think it’s pretty clear what’s going on here. In short, we set the environment variables LC_ALL and LANG because as we can read here Fastlane requires this. We create a new build when changes are made to the branch master, develop, and feature branches. We use a runner which is tagged with ios tag. We use the Fastlane build command and we store the result from the build directory for 90 days. After pushing this file to our git repository on Gitlab.com website in the CI / CD section we should see that our project is building. After successfully completing the build job we will see a green status icon and we will be able to download the build artifacts. If our build failed, we check in the log what went wrong. If we installed RVM on our build agent, then most likely in log we will see: ERROR: Job failed: exit status 1. To solve this problem, we have to edit the .bash_profile file on our build agent by adding the “unset cd” line to this file. We can do that from the terminal with the following command:

echo 'unset cd' >> ~/.bash_profile

After that everything should work fine also if we use RVM.

Over-The-Air IPA Installation from Gitlab

So we configured Gitlab CI and we have a successful build on Gitlab.com. As you may have noticed, we can download the IPA file from the Gitlab website, but it would be useful to have a possibility to install a build directly from Gitlab on our iPhone or iPad. To install the IPA file on iOS in addition to the IPA file, we need a special manifest.plist file and an HTML page with a proper link to this manifest file. Build artifacts on Gitlab are compressed in a zip file, therefore, it’s impossible to install IPA on our device directly from Gitlab. But here comes the Gitlab Pages service. In short, Gitlab Pages is simply a web hosting for static files which we manage through Gitlab CI. There is no possibility to run PHP scripts, only pages in HTML and other static files can be hosted there. That’s exactly what we need. For the OTA installation, we need manifest.plist file, so modify the Fastfile to generate it in the following way:

default_platform(:ios)
platform :ios do
  desc "Build application"
  lane :build do
    gym(
      scheme: "TestProject",
      clean: true,
      output_directory: "public",
      export_options: {
        method: "development",
        manifest: {
          appURL: "https://yourusername.gitlab.io/testproject/TestProject.ipa",
          displayImageURL: "https://imgplaceholder.com/57x57/000/fff/glyphicon-download-alt",
          fullSizeImageURL: "https://imgplaceholder.com/512x512/000/fff/glyphicon-download-alt"
        }
      }
    )
  end
end

Here we’ve added a part related to the manifest file and changed the name of output directory to public. Gitlab Pages requires the use of a directory named public, so that is the reason for that change. Note that in appURL you have to edit your username and the name of the project to match your project on your account. Generally, as you can see from this link, the page on Gitlab Pages for each of your projects has URL https://yourusername.gitlab.io/projectname/. Last needed file, that is HTML file with the link to the manifest.plist file we can just put on our git repository. So we create a new directory named public and in it, we create a file index.html with the following content:

<!DOCTYPE html>
<html>
<head>
  <meta content="width=device-width" name="viewport" />
  <title>TEST PROJECT</title>
  <style>
    body { text-align:center }
    a { color: firebrick; text-decoration:none }
  </style>
</head>
<body>
  <h1>TEST PROJECT</h1>
  <h2><a href=itms-services://?action=download-manifest&url=https://yourusername.gitlab.io/testproject/manifest.plist>⤓ DOWNLOAD</a></h2>
  <h3>{time}</h3>
</body>
</html>

This simple HTML with a link to the manifest will allow us to install the IPA file directly from Gitlab. In the above HTML code, you just need to edit the link to the manifest file as you did with the Fastfile. As you may have noticed I placed a placeholder {time} in that HTML code. During the publication to Gitlab Pages, we will replace it with the current time.

Next, we modify the .gitlab-ci.yml file to have the following content:

before_script:
  - export LC_ALL=en_US.UTF-8
  - export LANG=en_US.UTF-8
stages:
  - build
  - publish
build_job:
  stage: build
  tags:
    - ios
  script:
    - fastlane build
  artifacts:
    expire_in: 90 days
    paths:
      - public
  only:
    - master
    - develop
    - feature/*
pages:
  stage: publish
  script:
    - sed -i -e "s/{time}/$(date '+%Y-%m-%d %H:%M:%S')/g" public/index.html
  artifacts:
    expire_in: 90 days
    paths:
      - public
  only:
    - master

We’ve added a new stage and the pages section. So what this section does? First, it will replace {time} placeholder with the current date and time, then the entire contents of the public directory, that is index.html, IPA file built in the previous stage, and also manifest.plist file will replace the previous content on Gitlab Pages server. Uploading to Gitlab Pages will only take place for builds on the master branch. After git push, when our build is ready and successful, we can open Safari on our iPhone or iPad and go to https://yourusername.gitlab.io/testproject/. From there we can install the IPA file on our device. If your build passed successfully on Gitlab but the link does not open, then probably you just have to wait a moment because Gitlab Pages was not initialized yet for your project. Before the website on Gitlab Pages starts working for the first time, it may take about an hour or more. However, if your website starts working then subsequent builds should refresh on Gitlab Pages immediately.

Unfortunately, currently Gitlab Pages does not offer any method of authorization and anyone can open our website if only they know the correct link. It also does not support authorization using .htaccess and .htpasswd files. If we need some additional protection, we can put our IPA file and manifest file in some additional subdirectory so that the link is much more difficult to guess. For example, we can modify the index.html file to the following form:

<!DOCTYPE html>
<html>
<head>
  <meta content="width=device-width" name="viewport" />
  <title>TEST PROJECT</title>
  <style>
    body { text-align:center }
    a { color: firebrick; text-decoration:none }
  </style>
</head>
<body>
  <h1>TEST PROJECT</h1>
  <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.2/rollups/sha1.js"></script>
  <script>
    var key = prompt("Key:", "");
    var hash = CryptoJS.SHA1(key);
    document.write("<h2><a href=itms-services://?action=download-manifest&url=https://yourusername.gitlab.io/testproject/", hash, "/manifest.plist>⤓ DOWNLOAD</a></h2>");
  </script>
  <h3>{time}</h3>
</body>
</html>

Then when we open our page, we will be asked for a password and a hash of this password will be used as the name of the directory in which we will place the IPA file and the manifest file. Of course, we must also edit paths in the Fastfile file by including the appropriate hash. This is not an ideal solution, however, access to our IPA file by other people without knowing the correct hash will be more difficult.

EDIT 2020:

Recently Gitlab added a new functionality to control the access to Gitlab Pages. Now on the project page, you can select the Settings tab and then you can expand “Visibility, project features, permissions” section. In this section there is a new option related to Gitlab Pages and you can switch the access from “Everyone” to “Only Project Members”. After that only project members logged on Gitlab will be able to open your website.

Conclusion

Thanks to Gitlab.com we have our completely free CI and we have learned something about Fastlane tool. We also managed to create a mechanism for installing IPA straight from Gitlab, so we do not need to use any additional services and we have everything in one place. Of course, if we need something more advanced to publish our application, with the help of Fastlane we can easily add for example integration with Testflight. I encourage you to learn more about Fastlane and adapt the presented solution to your needs.