Continuous Delivery to Test Flight with Ionic and Cordova on CircleCI without Fastlane

It’s a stormy summer Sunday in Minneapolis as I do this write-up on an interesting engineering challenge I was handed this week. Namely, integrate CI and CD for an Ionic app with delivery to Testflight for beta users.

Here at Tarmac.io we’ve been doing full-on CI and CD for Native mobile (iOS and Android) as well as React-Native for years. Our customers have become dependent on the frequency of builds and continuous iterative feedback cycle that leads to better products, less confusion, and happier users. We’ve developed a well-worn track for delivering these CI and CD solutions using the usual suspects like CircleCI, Fastlane, HockeyApp, and Testflight. In fact we’ve gotten so quick at that setup that we’re typically doing fully autonomous CD within a few hours of generating the distribution certificates and provisioning profiles.

So why do it any other way? Great question :). Sometimes when we are handed lemons we need to figure out how to make lemonade anyways. Which brings me to this particular situation.

It is a long story, but if you’ve been writing software for any length of time you’ve heard a version of it. Customer has an app that they’ve just discovered is miles away from what they and their clients expect (and we’ve just inherited). We’ve now got 2 months to get this sucker back on track. Do you think we’re going to disappear for 2 months and cross our fingers that we’re better developers so “we’ll just get it right”? Hell no! This is the classic example of why CD is so important.

We’re going to retool the whole delivery workflow from the product management team through production launch!

This means that we are shifting from a “go away and build it” to a strategy that involves building one thing at a time with continuous feedback from the customer — this is always the winning strategy.

The requirements for CI and CD are these:

• Ionic and Cordova 5
• CircleCI 2.0
• Cannot use fastlane match (I am crying now)
• Cannot use fastlane
• Keep platforms directory (the one generated by cordova) out of git source control
• Build and sign with Apple app store distribution certificate and app store provisioning profile
• Upload to Testflight
• No human intervention

Brutal list but I’m stubborn and up for the challenge. Regarding the non-use of fastlane — I found that the ionic and cordova plugins for fastlane are severely lacking especially related to the signing aspect of the build process. All the research I’ve done to date use these plugins but then resort to some level of manual interaction to get the build signed and uploaded to the store. That is a no go for us so there is some hand-rolling of the process to be expected here.

I need to note that any that knows me knows how much I hate hand-rolled solutions, but if we need to live with one to fully automate — the tradeoff seems appropriate

Let’s get started…

Without fastlane match we need to do a couple things to prepare CircleCI for our work.

1. In Apple Developer generate your App Store Distribution certificate request, upload it, and download the resulting certificate file.
2. Import the .cer file into your own keychain by double clicking the downloaded file and entering your password (if prompted).
3. Using Keychain access find the imported certificate, highlight it, and export it as a .p12 file — password protect it if you want.

We’re now going to base64 encode that .p12 file and set it as an environment variable in CircleCI called DISTRIBUTION_KEY.

$ base64 -i Certificates.p12 | pbcopy

Next generate an app store provisioning profile and download it. For the purposes of this project I created a directory in my project called provisioningprofiles and I am checking them into the repo with source control. Though you could use the base64 technique like above and set them as an ENV variable in CircleCI to be written to the filesystem later. This is your call.

Signing

Next we’ll add some steps to the .circleci/config.yml to install our certificates and ensure permissions are set correctly for the build steps.

- run:
          name: decode Certificates
          context: org-global
          command: base64 -D -o ~/cert.p12 <<< $DISTRIBUTION_KEY

This step creates a .p12 file with the base64 decoded key from the environment variable we set above.

Now let’s create a new keychain (we can’t use circleci login keychain for this) and install this certificate for signing.

The step above does a bunch of things; I’ll summarize.

• Create a new keychain called MyKeychain
• Unlock the keychain so we can modify its properties and add the signing key
• Import the .p12 file
• Increase the timeout from the default of 300 seconds (5 minutes)
• Add the new keychain and link the previous keychains
• Set the default keychain to the newly created keychain
• Print the results so you can see them in CircleCI

Next we’re going to move the Provisioning Profile from the repository to the appropriate location on the build server.

A bit of oddness I discovered on CircleCI MacOs images. Sometimes this directory and the files I move get created with the root user. I couldn’t find a pattern so I brought out the chown hammer and just made sure that we are always set with the correct permissions.

Next, lets install ionic, cordova, and yarn

- run:
          name: "Install ionic and cordova"
          command: |
            sudo npm install -g ionic cordova yarn
      - run:
          name: "Install npm packages"
          command: |
            sudo chown -R distiller:staff ../.config
            yarn install

We’ll now install any plugins we may need to prep for the iOS release build.

- run:
          name: "Install Cordova plugins and add ios"
          command: |
            sudo chown -R distiller:staff ../.npm
            ionic cordova plugin add cordova-ios-plugin-no-export-compliance
            ionic cordova platform add ios --noresources
            ionic config set -g telemetry true

I found some real inconsistencies with permissions on files and directories on circle ci. Again I bring out the chown hammer and make sure that the npm directory for the distiller user is owned by our build user.

We add the cordova-ios-plugin-no-export-compliance to ensure a truly hands free deployment. Without this plugin your builds will wait this compliance answer in testflight before your users can see it. Add the plugin — assuming of course you aren’t exporting encryption :).

Versioning

iTunesConnect, Apple, and Testflight require unique Version and Build numbers. These must be incremented by hand — oops that doesn’t meet our requirements… Luckily this is controlled via the config.xml file in the root of the project. Add this directive to the widget definition (line 2)

version="0.0.1" ios-CFBundleVersion="SET_BUILD_NUMBER"

We’re now going to use some command line fu to grab the build number from circle ci and set the bundle version so it is always unique.

- run:
          name: "Increment iOS build number"
          command: sed -i '' 's/SET_BUILD_NUMBER/'"$CIRCLE_BUILD_NUM"'/g' config.xml

Build

Finally we’re going to build the .ipa file just before sending it off to testflight

Use chown to ensure that the ionic directoy (in the distiller user home) is set appropriately for all the files.

I want to point out this next step

security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k default /Users/distiller/Library/Keychains/MyKeychain.keychain-db

This is critically important. Without this step the build will never sign successfully and you will be stuck forever with the following error message.

errSecInternalComponent
Command /usr/bin/codesign failed with exit code 1

and

xcodebuild: Command failed with exit code 65
[ERROR] An error occurred while running subprocess cordova.

If you are seeing these errors even with an unlocked keychain, and you aren’t bumping into the keychain unlock timeout, then it is because you have not appropriately set the context for the codesign action on this keychain. This is a very obscure thing but your builds will fail without it, and worse they will pass locally every single time.

Ok, time to make the build. Note the build.json — some very valuable information needs to go here. It looks like this and it lives in the root of the project.

Get the guid for the provisioning profile by opening the profile and looking for the entry called UUID its hexadecimal. Get the team ID from apple developer, the key, your keychain, its a bunch of places.

Deploy

Finally, I cheated and delivered to Testflight using fastlane. I got sick of doing this by hand. It looks like this.

- run:
          name: "Upload to test flight"
          command: fastlane upload_build

So easy.

The lane

lane :upload_build do
    upload_to_testflight(
      username: ENV["APPLE_USER"],
      apple_id: ENV["APPLICATION_ID"],
      app_identifier: "YOUR_APP_IDENTIFIER",
      skip_waiting_for_build_processing: true,
      ipa: "PATH_TO_IPA"
    )
  end

To wrap it up. Do I love this solution? HELL NO! Does it get us into CD using this toolset, yes it does. In this case I think the ends will justify the means for now. We’ve managed to keep the cordova iOS project generation completely clean without checking it into the repo and have met the other requirements as well.

I would much prefer a standard toolset every single day of the year. My hope is that this post might help the next person struggling with a similar set of issues. Feel free to leave me a note below, if I can help I will.

~Brent Kastner