How Fundrise Freed Itself from Xcode Project Purgatory
The Problem with Xcode Project Files
It’s 4:35 pm on a Friday. You’ve just finished your iOS masterpiece feature. To release it, you need to merge from main and push. But when you attempt your git merge, you see the most dreaded console message in any iOS developer’s life: CONFLICT (content): Merge conflict in project.pbxproj. Defeated, you shut your laptop lid and go home, ready to fight with the project file on Monday.
This is a story every iOS Developer knows all too well. Checking the Xcode project into source control is one of our biggest problems to work around. Even the slightest change can cause problems, including routine changes like moving or deleting files. Looking at the history in git, it’s very hard to make sure that the changes that were made are limited to changes you made intentionally.
Xcode will arbitrarily move lines, change settings slightly in meaningless ways, and generally add a lot of noise to otherwise benign Xcode project changes. This gets so bad, that many developers try to minimize any changes to the project regardless of the need. The project quickly stops matching the filesystem, dependencies go un-upgraded, and changes then happen in big moves to limit the pain.
When our team was smaller our first defense was to add *.pbxproj binary merge=union to our .gitattributes. This solves a lot of minor conflicts and issues, but would occasionally break the project file, sometimes beyond repair. Many iOS engineers have become experts in editing the pbxproject file in plain text. As one of these unfortunate engineers, I can testify that it is not a fun experience. These errors could easily derail one or more engineers for a day or more.
Why Tuist?
But we’re stuck, right? Apple requires the use of Xcode, and Xcode uses the pbxproj format. But all is not lost, as there are a number of projects, like XcodeGen, Tuist, and Tulsi, that will automatically generate the project file.
Here at Fundrise, we chose Tuist for a number of reasons. It is written in Swift, which allows our iOS developers to contribute easily, as Swift is a language they are all familiar with. It has an extensible architecture that allows for building, plugins, code generation, and anything else you can do with Swift. It also allows us to manage our dependencies using Swift Package Manager, and speed up our builds by pre-building our dependencies and caching their binaries.
As we scale the codebase, Tuist is also built to allow for easy modularization. Before we moved to Tuist, we had been using CocoaPods to do dependency management. Tuist does not currently support CocoaPods directly so we made the decision to move to Swift Package Manager simultaneously.
The Path to Project File Removal
If you can switch to Swift Package Manager first, or never involve CocoaPods at all, that’s your best bet. CocoaPods makes its own changes to the project file that can cause a number of issues. In another article, we will cover making this switch.
In order to prepare your projects for Tuist generation, every setting in the project needs to be defined outside the actual project itself. Tuist recommends removing all your settings from being set in the Xcode Project. This can be done by extracting all of your project settings into xcconfig files.
Tuist offers a built in migration tool, but we decided to use James Dempsey’s Build Settings Extractor as it pulls out all of the settings in a hierarchical format organized by target, and offers optionally commented exports to help you make sense of the settings it exports. An invaluable resource if you want to understand xcconfigs is the page Xcode Build Settings. It’s a searchable list and explanation of Xcode build options.
Xcconfigs can be fragile, so you’ll have to use caution. One common, dangerous example is Xcode treating all double slashes, even in strings, as the start of an xcconfig comment. This will cause all URLs that don’t use escaping to parse as the plain string “https:’’ with no data after the colon. Our solution was to put a blank variable escape in URLs like this: “https:/$()/example.com”. As you move settings, you’ll want to keep an old copy of your project around to make sure that you haven’t changed anything inadvertently. A tool like xcdiff can help compare your project.
There are a number of pitfalls you’ll want to avoid as your team makes the switch. You’ll need be certain everything is explicitly set. Shell scripts should be pulled out of the project file into files and checked into source control. You’ll be able to set source directories using file globs, but Tuist will only pull in source files. Anything that needs to be in the project that is not a source file needs to be added explicitly. Fonts, assets, JSON, HTML, xcconfigs, entitlements, plists, and anything and everything that doesn’t end in *.swift or *.m will need to be added.
You’ll also need to be on guard since bugs from files that are missed or added improperly can cause strange behavior. For instance, if the filesystem has files that were deleted from the project but not the filesystem, those will be added and the compiler will attempt to compile them. In our case, this led to us discovering dead files that would not compile due to being very out of date.
Tuist is also an active project, and you may run into some bugs in Tuist itself as you transition. Tuist’s slack workspace is active, and the project is responsive to well described issues. For instance, we found two bugs relating to our project, one that involved core data models of a dependency and another that involved Info.plists. Both were fixed quickly by the Tuist team and were able to be incorporated into our setup before we rolled it out to the team.
Finally, as you begin to move your project over, you will run into what I call “The Rollout Paradox.” You’ll want to minimize the changes to your project to make sure that you are indeed building the same thing, but Tuist has sensible defaults and it can be best not to fight it. We found that the easiest method was to base the structure of the project around Xcode Targets. Tuist’s generated targets include sensible recommended and default configurations. Our project used schemes heavily for build time configuration, but we moved this logic into targets. This allowed us to automatically generate all of our schemes, eliminating the need for manual maintenance.
Lastly, like all things in software engineering, this will take longer than you expect. The Xcode project has a lot of hidden settings, hidden corner cases, and otherwise rough edges that will cause you migration problems. However, once the rollout happens, the payoff is immediate.
The Rollout
When rolling out the new project, you’ll want to take your time. We staged our rollout right after a major release, and held office hours to make sure that everyone could install and run Tuist. After you delete the project from source control, any branch that changed will have a conflict. You’ll both need to educate developers on how to resolve it, and add a check to your CI so builds will fail if anyone tries to check in any project files.
The first sign of improvement came on the second day after delivery. An engineer came to me for help updating a long-neglected branch with a lot of file changes. This branch was nearly a thousand commits behind and included major conflicts in the project file. In the past, this would have required hours of painstaking work to repair. But we were able to quickly fix it by deleting the project file and running the tuist generate command.
The second improvement we noticed was the project structure being clearly and explicitly defined as it was generated from the filesystem. It was now impossible for our project to contain any orphaned files. This is critical to maintaining consistency in our project as it grows.
Where to go from here?
As we rolled out Tuist, we identified a number of ways that we could improve our workflows. For example, to avoid adding things like “https:/$()/example.com” in xcconfigs as a workaround is using Tuist’s ability to generate code. These environment based settings can be placed into code where they can be tested and type checked.
We’re looking to increase productivity and build speed with each iteration of our Tuist upgrades. One step is to begin modularizing the project, which Tuist makes easier and less disruptive. Another initiative is to begin caching our dependencies. Finally, we want to investigate tuist plugins to speed our development process.
Conclusion
And now, thanks to Tuist, on Friday, I close my laptop confidently and walk off into the sunset, having shipped that groundbreaking new feature. If you’re interested in working with talented iOS engineers on problems like this, enabling us to scale our codebase and engineering teams, check out our careers page at https://fundrise.com/jobs.
