Speeding up Carthage for iOS Applications

Using Make, Git LFS

Update: I wrote a response on how to use Artifactory to better track the Archives which avoids tracking them in Git.

Carthage is a nice framework dependency manager. It is simple, very well defined and works as documented. For the contract tests demo post that I wrote a while back, the Xcode project has about 7 Swift files (3 are Sources, 4 are Tests) and 5 dynamic frameworks. A clean build in Travis CI took close to 14 min. This was ludicrous.

Then I began another iOS project, this time with quite a few frameworks and Carthage in two. The build times are atrocious. Over 40min was spent just compiling the frameworks.

Thus began the search to optimize it. There are very few alternatives and not much documentation or guides to work with.

The easiest option was to track the “Carthage/build/iOS/” directory in source control but “*.framework” files have a rather interesting structure which is not SCM friendly.

Archiving Carthage Frameworks

Carthage has an interesting command called archive which allows for archiving a framework with the dSYM files.

carthage help archive

That gave me an idea, if we could list all the frameworks built in the “Carthage/build/iOS” directory, we could run carthage archive on those

1. Listing all the Carthage Frameworks

ls Carthage/Build/iOS/*.framework | grep "\.framework" | cut -d "/" -f 4 | cut -d "." -f 1

As we were already using make, it was easy to run shell commands. So creating, the archives of all frameworks that Carthage built became

CARTHAGE_FRAMEWORKS=ls Carthage/Build/iOS/*.framework | grep "\.framework" | cut -d "/" -f 4 | cut -d "." -f 1 | xargs -I '{}'

carthage_update: ## update carthage packages
  carthage update --platform iOS --no-use-binaries

carthage_archive: carthage_update ## update and archive carthage packages
  rm -rf PreBuiltFrameworks/*.zip
  $(CARTHAGE_FRAMEWORKS) carthage archive '{}' --output PreBuiltFrameworks/

The above make target allows for all Carthage frameworks to be archived with all debugging symbols to the PreBuiltFrameworks directory. It also updates the Carthage frameworks before archiving them.

Using the Archives

Well, that was easy but we still need to use them.

CARTHAGE_ARCHIVES=ls PreBuiltFrameworks/*.zip | grep "\.zip" | cut -d "/" -f 2 | cut -d "." -f 1 | xargs -I '{}'

carthage_extract: ## extract from carthage archives
  $(CARTHAGE_ARCHIVES) unzip PreBuiltFrameworks/'{}'.framework.zip

The carthage_extract target does the reverse of the carthage_archive target, extracting all the frameworks to the directory where Xcode expects them to be.

Tracking the Archives with Git LFS

Building the archives is easy and so is extracting them to use. The next step is tracking them efficiently in source control. Git is bad at tracking large binary files, bloating the repository size and thus making the git operations slower.

Git LFS is much better at large binary files while keeping the Git repository faster and performant.

1. Installing Git LFS

brew install git git-lfs
git lfs install

2. Using Git LFS to track Archives

The caveat with Git LFS is that we need to ask it to track each archive, it doesn’t yet have support for wildcards.

git lfs track PreBuiltFrameworks/*.zip
git add .gitattributes
git commit -m "Tracking Prebuilt Frameworks"
git add PreBuiltFrameworks/*.zip
git commit -m "Adding Prebuild Framworks"

This will have to be done everytime a new framework gets added. It can be automated as a make target if you need to.

carthage_track: ## track and commit carthage frameworks
	git lfs track PreBuiltFrameworks/*.zip
	git add .gitattributes
	git commit -m "Tracking Prebuilt Frameworks"
	git add PreBuiltFrameworks/*.zip
	git commit -m "Adding Prebuild Framworks"

Some Gotchas

Git LFS support is somewhat weak in most CI/CD tools. Jenkins does not work or at least I don’t know how to make Git LFS work with it. GoCd requires that git lfs is installed and configured.

For Travis CI, here’s the .travis.yml that I use

language: objective-c
osx_image: xcode8
cache:
  - bundler

before_install:
- brew install git-lfs
- git lfs install
- git lfs pull
- make install_bundle carthage_extract

script:
- make test

after_script:
  - bash <(curl -s https://codecov.io/bash)

Extras

One of the setup tasks for Carthage is to run carthage copy-frameworks in a Run Script phase on Xcode. It is a lot of typing (while taking care of typos) to write each and every framework used. Now we can use a make target like this to do that instead

CARTHAGE_FRAMEWORKS=ls Carthage/Build/iOS/*.framework | grep "\.framework" | cut -d "/" -f 4 | cut -d "." -f 1 | xargs -I '{}'

carthage_copy: ## copy carthage frameworks
  $(CARTHAGE_FRAMEWORKS) env SCRIPT_INPUT_FILE_0=Carthage/build/iOS/'{}'.framework SCRIPT_INPUT_FILE_COUNT=1 carthage copy-frameworks

In Xcode, the Run Script phase can now just do

cd "$SRCROOT"; make carthage_copy

Combining it all

Here’s the snippet of the makefile with all the relevant targets. A fuller makefile is here.

CARTHAGE_FRAMEWORKS=ls Carthage/Build/iOS/*.framework | grep "\.framework" | cut -d "/" -f 4 | cut -d "." -f 1 | xargs -I '{}'
CARTHAGE_ARCHIVES=ls PreBuiltFrameworks/*.zip | grep "\.zip" | cut -d "/" -f 2 | cut -d "." -f 1 | xargs -I '{}'

.DEFAULT_GOAL := help

install_bundle: ## install gems
  bundle install

install_carthage: ## install carthage frameworks
  carthage bootstrap --platform iOS --no-use-binaries

install: install_bundle install_carthage ## Install Gems, Carthage

carthage_clean: ## clean up all Carthage directories
  rm -rf Carthage

carthage_update: ## update carthage packages
  carthage update --platform iOS --no-use-binaries

carthage_archive: carthage_update ## update and archive carthage packages
  rm -rf PreBuiltFrameworks/*.zip
  $(CARTHAGE_FRAMEWORKS) carthage archive '{}' --output PreBuiltFrameworks/

carthage_track: carthage_archive ## track and commit carthage frameworks
	git lfs track PreBuiltFrameworks/*.zip
	git add .gitattributes
	git commit -m "Tracking Prebuilt Frameworks"
	git add PreBuiltFrameworks/*.zip
	git commit -m "Adding Prebuild Framworks"

carthage_extract: carthage_clean ## extract from carthage archives
  $(CARTHAGE_ARCHIVES) unzip PreBuiltFrameworks/'{}'.framework.zip

carthage_copy: ## copy carthage frameworks
  $(CARTHAGE_FRAMEWORKS) env SCRIPT_INPUT_FILE_0=Carthage/build/iOS/'{}'.framework SCRIPT_INPUT_FILE_COUNT=1 carthage copy-frameworks

help:
  @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'