From Fat Frameworks to XCFrameworks, or how to support arm64 for iOS Simulators

Hello!

My name is Alexey Khalaydzhi and I’m engaged in the development and maintenance of the infrastructure for mobile apps development. I develop and implement new tools for programmers, research DevEx optimization techniques, beginning from what happens on the developer’s machine to automatic release rollout and application usage monitoring for real users. Sometimes people ask me to audit the infrastructure of existing mobile projects and suggest specific solutions to improve it.

Last summer, I encountered some unusual problems in one of iOS-projects, given to me for such an audit — developers claimed their app began to freeze massively (it stopped responding to any UI interactions). The problem was reproducing during debug on the simulator right after launch. Moreover, after we updated the Xcode version from 13.2.1 to 13.4.1, the built-in debugger became inoperable. This hindered the development process. What caused these problems, how it became possible to understand what was the matter, and how everything was fixed — all of this will be discussed in the article.

Beginning of the story

There were many hypotheses about what could go wrong, when this problem appeared for the first time. Here are just a few of them:

- breakdown could be related to the update to the new Xcode version;

- something could become broken in app’s launch logic (because not the whole simulator hung, but only app itself after launch, when many concurrent operations were running, including networking);

- the project used XCRemoteCache to speed-up build time and developers had already faced some debugging troubles, using this tool, that lied in breakpoints accidentally stopped working — so I also had some suspicions about it.

The problem was aggravated by the fact that it was not reproducible by everyone and at the first time — sometimes, one had to run the app dozens of times to catch the freeze. Moreover, this, strictly speaking, could not be called a “hangup”, because there was a spinner in the UI, indicating loading was in progress, but the app did not react to any other events in the UI. At the same time, when trying to pause the application and advance the main thread via debugger, the latter crashed, although there were no such problems with other threads. The latter suggested even more that developers might have accidentally done something wrong with multithreading, that caused the main thread to be blocked. However, without a debugger connected, and on real devices, the problem was not reproducible.

After a week of unsuccessful tests of the above hypotheses, studying documentation from Apple, discussions at forums and checking other resources — still a solution was not found yet. However, I managed to find another indirect problem — with such a hang, restarting neither the application nor the simulator itself helped. When trying to profile the application, it was clear that the main thread really hangs, but on symbols from runtime, when there weren’t any direct calls from the application in the stack traces.

Upon a more detailed investigation of this problem, I found that the application process just cannot be completed as usual, it even survives the removal of the application from the simulator. However, in all such cases, the hung process always had a parent debugger process:

/Library/Apple/usr/libexec/oah/debugserver

The destruction of the debugger process led to the termination of the hung process, and this allowed to solve the problem symptomatically. However, while I was dealing with the reproducibility and symptoms investigation of the problem, the number of complaints from developers about it increased significantly day after day. As a result, that problem escalated to blocker priority.

In parallel with the analysis of what is wrong with the simulators, I investigated the work of the debugger — the usual p and po command did not work at all, and it was possible to use only the v command to output the value of a variable from the current stack frame. All of these commands were discussed with more details at WWDC’19 (LLDB: Beyond “po”). At the same time, when returning to Xcode 13.2.1 or updating to Xcode 14 beta, the debugger could be used without any problems. However, if the transition solved the problem with the debugger, neither updating to the new beta version of Xcode, nor even updating macOS to the beta version of Ventura helped with the hang problem.

At least, I discovered that on macbooks with M1 chip, when debugging the app, a similar message appeared in the console:

Warning: Error creating LLDB target at path ‘…/foo.app’- using an empty LLDB target which can cause slow memory reads from remote devices: the specified architecture ‘arm64-*-*’ is not compatible with ‘x86_64-apple-ios11.0.0-simulator’ in ‘foo.app/foo’.

Among other things, it was such a time period, when every day more and more developers were replacing their macbooks on Intel with macbooks on M1 chip, because the latter worked faster (in particular, app was being built much faster), warmed up less, and, at least, market suggested more and more macbooks on new architecture. Once again, I gathered all the clues together and formulated a new hypothesis — the problem was reproduced only on M1 macbooks, and the increasing number of complaints was primarily due to an increase in the number of developers working on such device models.

Now it is the appropriate moment for a small digression to explain the fundamental difference between macbooks on Intel and macbooks with M1 chip for iOS developers. iOS Simulator is a MacOS app — that’s why it launches on the same architecture, OS is running on. It was x86_64 architecture for Intel and arm64 for M1. And everything would be fine, because arm64 is not a new architecture, after all, because all the latest models of Apple mobile devices are actually based on it. That is why iOS apps are mostly often assembled immediately for at least two architectures: x86_64 (for simulators) and arm64 (for real devices), and depending on the specific model of the final platform, one or another version of the application is chosen and actually being installed.

However, it turned out that the versions of the arm64 frameworks, used to build applications on real devices, were not suitable for building simulator apps. When attempting such an assembly (and on M1 the assembly is performed for arm64 architecture by default), an error with a similar message is usually returned:

… foo.framework/foo building for iOS Simulator, but linking in object file built for iOS… for architecture arm64.

Most often, iOS developers community members suggest to deal with that error by excluding this architecture for simulators by setting the following project property (and also in Podfile or other management system configuration file for all pluggable dependencies):

EXCLUDED_ARCHS[sdk=iphonesimulator*] = "$(inherited) arm64"

It also worked in my case when I encountered this problem for the first time. According to that setting, the simulator app should be always built for all supported architectures besides arm64 — that is only x86_64 at that moment. However, at the same time, there was a little misunderstanding in my mind about how the final app, built for x86_64, could be actually launched at all without any additional actions on the iOS Simulator, which was running under the OS with arm64 architecture.

It turned out that in all such cases the application process is implicitly virtualized via Rosetta. To understand in detail, what exactly is going on under the hood, I recommend you to read the following papers: How does rosetta 2 work, Running Intel code on your M1 mac: Rosetta 2 and oah и Project Champollion.

In listed sources you can find, that Rosetta uses executable files located in the directory /Library/Apple/usr/libexec/oah — the same directory where debugserver process is located — the same process file, that was launched in my case during debugging and that kept the application hanging.

$ ls -la /Library/Apple/usr/libexec/oah
total 328
drwxr-xr-x 7 root wheel 224 23 Aug 03:26 .
drwxr-xr-x 4 root wheel 128 23 Aug 03:26 ..
drwxr-xr-x 3 root wheel 96 2 Aug 15:06 RosettaLinux
lrwxr-xr-x 1 root wheel 32 23 Aug 03:26 debugserver -> /usr/libexec/rosetta/debugserver
-rwxr-xr-x 1 root wheel 365168 1 Aug 10:37 libRosettaRuntime
lrwxr-xr-x 1 root wheel 28 23 Aug 03:26 runtime -> /usr/libexec/rosetta/runtime
lrwxr-xr-x 1 root wheel 35 23 Aug 03:26 translate_tool -> /usr/libexec/rosetta/translate_tool

This only strengthened my faith in the hypothesis, that the hang was reproducible only on M1 chips and only under debugging, somewhere on some strange characters somewhere inside the runtime. Immediately an assumption appeared about a possible solution — try to honestly support app building for arm64 on iOS Simulators.

Approaches for arm64 support for iOS-simulators

When developer has the full source code of the entire project, and it does not use platform-dependent solutions for Intel (for example, similar ones may be met in bio authentication applications, etc., where some direct work with the low-level hardware API of the device is going one) — there should be no problems with arm64 support. However, more often developers use external binary dependencies (i.e. precompiled libraries/frameworks without access to the source code), and still not all of them might support arm64 for simulators. In such a case, it’s recommended to contact the maintainers of these dependencies and ask them to support that architecture. However, even this is not always possible — for example, due to the presence of legacy dependencies that are no longer supported.

How to understand that a particular framework supports the right architecture? There are several ways to get an answer to this question. To better understand all further explanation, I prepared distinct repository with sample demo project on Github — feel free to play with it and check all commands there while reading this paper. There are two branches in that repository: main contains a project without any xcframework and reproduces initial problem with absence of support for arm64 architecture when building on simulators, when xcframeworks branch contains the result project with provided support as will be described further.

Source project is provided with the following Podfile:

platform :ios, '11.0'
use_frameworks! :linkage => :static # prefer static frameworks
target 'RouteToARM64SimSampleProject' do
  pod 'libyuv-iOS'   # static fat lib
  pod 'MNN'          # binary fat static framework
  pod 'mob_sharesdk' # binary fat dynamic framework + static framework
end

To prepare a project, one should execute pod install command in the terminal at the project’s folder. After that one can verify that the dependencies have different architectures using the following tools: file, xcrun vtool and lipo. Also, one could use the otool utility, but its output is too cumbersome, so it will not be shown in this article.

Look at the output of the file tool:

$ file Pods/libyuv-iOS/lib/libyuv.a
Pods/libyuv-iOS/lib/libyuv.a: Mach-O universal binary with 4 architectures: [arm_v7:current ar archive random library] [i386:current ar archive random library] [x86_64:current ar archive random library] [arm64:current ar archive random library]
Pods/libyuv-iOS/lib/libyuv.a (for architecture armv7): current ar archive random library
Pods/libyuv-iOS/lib/libyuv.a (for architecture i386): current ar archive random library
Pods/libyuv-iOS/lib/libyuv.a (for architecture x86_64): current ar archive random library
Pods/libyuv-iOS/lib/libyuv.a (for architecture arm64): current ar archive random library

$ file Pods/MNN/MNN.framework/MNN
Pods/MNN/MNN.framework/MNN: Mach-O universal binary with 3 architectures: [arm_v7:current ar archive] [x86_64] [arm64]
Pods/MNN/MNN.framework/MNN (for architecture armv7): current ar archive
Pods/MNN/MNN.framework/MNN (for architecture x86_64): current ar archive
Pods/MNN/MNN.framework/MNN (for architecture arm64): current ar archive

$ file Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK: Mach-O universal binary with 5 architectures: [arm_v7:Mach-O object arm_v7] [arm_v7s] [i386] [x86_64] [arm64]
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (for architecture armv7): Mach-O object arm_v7
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (for architecture armv7s): Mach-O object arm_v7s
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (for architecture i386): Mach-O object i386
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (for architecture x86_64): Mach-O 64-bit object x86_64
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (for architecture arm64): Mach-O 64-bit object arm64

$ file Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation: Mach-O universal binary with 5 architectures: [arm_v7:current ar archive] [arm_v7s] [i386] [x86_64] [arm64]
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation (for architecture armv7): current ar archive
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation (for architecture armv7s): current ar archive
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation (for architecture i386): current ar archive
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation (for architecture x86_64): current ar archive
Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation (for architecture arm64): current ar archive

Here is the output of xcrun vtool instrument:

$ xcrun vtool -show Pods/libyuv-iOS/lib/libyuv.a
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/vtool error: Pods/libyuv-iOS/lib/libyuv.a file #0 for cputype (12, 9) is not mach-o

$ xcrun vtool -show Pods/MNN/MNN.framework/MNN
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/vtool error: Pods/MNN/MNN.framework/MNN file #0 for cputype (12, 9) is not mach-o

$ xcrun vtool -show Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (architecture armv7):
Load command 3
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 8.0
      sdk 15.4
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (architecture armv7s):
Load command 3
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 8.0
      sdk 15.4
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (architecture i386):
Load command 3
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 8.0
      sdk 15.4
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (architecture x86_64):
Load command 2
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 8.0
      sdk 15.4
Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK (architecture arm64):
Load command 2
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 8.0
      sdk 15.4

$ xcrun vtool -show Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation
/Applications/Xcode-14.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/vtool error: Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation file #0 for cputype (12, 9) is not mach-o

At least, when using lipo, you should see something like that:

$ lipo -info Pods/libyuv-iOS/lib/libyuv.a
Architectures in the fat file: Pods/libyuv-iOS/lib/libyuv.a are: armv7 i386 x86_64 arm64

$ lipo -info Pods/MNN/MNN.framework/MNN
Architectures in the fat file: Pods/MNN/MNN.framework/MNN are: armv7 x86_64 arm64

$ lipo -info Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK
Architectures in the fat file: Pods/mob_sharesdk/ShareSDK/ShareSDK.framework/ShareSDK are: armv7 armv7s i386 x86_64 arm64

$ lipo -info Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation
Architectures in the fat file: Pods/MOBFoundation/MOBFoundation/MOBFoundation.framework/MOBFoundation are: armv7 armv7s i386 x86_64 arm64

The results of execution of first two instruments, show that sample project uses 3 kinds of dependencies: static library (libyuv, uses archive of object files), static framework (MNN and MOBFoundation — similar to the previous one, but is a framework, that makes possible to bring some resources or some other assets additionally to the library itself) and a dynamic framework (ShareSDK — contains Mach-O binary file and is similar to dynamic libs, but actually is a framework). At the same moment, xcrun vtool utility failed for the most of dependencies, but not for ShareSDK, because it requires a Mach-O file as the input, when static libraries/frameworks are archives from such object files. For such archives the same information could be found without unarchiving via otool -vahl command, which will show everything in the same format for all object files, included in the archive.

Output of file utility shows that every dependency is actually so-called fat dependency — Apple calls them as universal binaries. That means, they bundle several architectures at once — you can find more details, for instance, from A deep dive on macOS universal binaries article. It would seem that it would be enough just to add another one — “arm64 for simulators” — and our initial problem would be solved. However, the main problem is that arm64 for the simulator will also be shown just as arm64, and it will conflict with existing one for real devices. So, one could make the following conclusion: two different versions of some dependency for the same architecture cannot coexist in one fat file.

To solve that problem, Apple suggested a new format for dependencies — xcframework (Binary Frameworks in Swift, WWDC’19).

What actually is a xcframework? Well, this is a package that allows you to deliver dependencies for different platforms simultaneously: iOS, tvOS, macOS, iOS Simulator, watchOS etc. The typical structure is as follows: at the root level there is an Info.plist file describing the internal structure of concrete xcframework, and there are many subdirectories with interpretive names — each of these subdirectories contains distinct (fat) framework/library, that provides support for all of the architectures for the concrete platform (platform and architectures are listed in the name of that subdirectory). For example, FirebaseAnalytics.xcframework looks as follows:

$ tree -L 2 FirebaseAnalytics.xcframework
├── Info.plist
├── ios-arm64_armv7
│   └── FirebaseAnalytics.framework
├── ios-arm64_i386_x86_64-simulator
│   └── FirebaseAnalytics.framework
├── ios-arm64_x86_64-maccatalyst
│   └── FirebaseAnalytics.framework
├── macos-arm64_x86_64
│   └── FirebaseAnalytics.framework
├── tvos-arm64
│   └── FirebaseAnalytics.framework
└── tvos-arm64_x86_64-simulator
    └── FirebaseAnalytics.framework

Many developers already support xcframeworks. However, as mentioned above, there are legacy dependencies that are no longer supported. In some cases, it’s possible to integrate the source code of such dependencies and assemble the necessary frameworks yourself, however, this is not always possible, too. Developers of the project, given to me for an audit, faced exactly such a case — they had about 200 external dependencies, and although it was theoretically possible to request access to the source code of all of them, it would be very inconvenient and take too much time. Therefore, they were interested in whether it’s possible to somehow patch the existing, already assembled libraries to add support of the desired arm64 architecture for simulators.

As it turned out, a solution exists! The article Hacking native ARM64 binaries to run on the iOS Simulator (and its second part — dynamic framework edition) describes in detail how exactly binary files, assembled for arm64 and for simulator at arm64, differ. In fact — not so much (as one should expect). Patch procedure differs for dynamic and static dependencies, but in both cases its main goal is to change the information about the target platform in the image loading section. Exactly that information about Mach-O files is actually shown when using xcrun vtool utility, as shown above.

To simplify the process of patching Mach-O files, the author of the listed above articles has developed a separate arm64-to-sim utility. It is installed quickly and simply from the source code (the full instruction is given in the repository’s README file). As input, the utility accepts a file that needs to be updated and two additional parameters — minos and sdk. The first one sets the minimum iOS version for which this dependency could be used, the second one — is the sdk version, used to build this dependency. By default, the utility puts the values minos=12, sdk=13. In fact, minos for arm64 simulators can be left equal even to 14, because it was the first version of iOS where such iOS Simulators appeared. It’s easy to make sure of this if you try to build an application on, for example, an iPhone X with iOS 13 — in such a case the application will start to build for the x86_64 architecture even on macbooks with M1. What about the second argument, sdk — well, Xcode 14 uses version 16.0, and it’s also easy to check with the following command:

xcodebuild -showsdks 2>/dev/null | grep -oE "iphonesimulator(\d+.\d+)" | sed -E "s/iphonesimulator//"

It’s quite possible to leave the default values (especially, since all changes relate only to simulator builds on arm64, without affecting the versions for the simulators on x86_64 architecture and, that’s more important, real devices on arm64). However, the most reliable way is to keep the same values as they were before the utility was running. For example, if the original load command looked like this:

Load command 9
      cmd LC_VERSION_MIN_IPHONEOS
      cmdsize 16
      version 9.0
      sdk 14.3

all needed information could be extracted via following commands:

VERSION=$(xcrun vtool -show -arch arm64 FILE_TO_UPDATE__REPLACE_ME | grep -oE -A3 "LC_VERSION_MIN_IPHONEOS" | grep -oE "version." | sed -E "s/version[[:space:]]//")
SDK=$(xcrun vtool -show -arch arm64 FILE_TO_UPDATE__REPLACE_ME | grep -oE -A3 "LC_VERSION_MIN_IPHONEOS" | grep -oE "sdk." | sed -E "s/sdk[[:space:]]//")

What does this patch utility actually do, why do we need that at all? Well, each platform has its own code in load command, and there are also several types of these load commands (you can find them in the original articles, listed earlier in this paper). So, for the iOS Simulator platform, the code 7 is used. There is one more nuance — the size of the iOS Simulator load command differs from the size of the load command for regular devices. In static object files that leads to the need of shifting all further content till the end of the file. In dynamic ones, the situation is somewhat simpler, because after their header section there is a buffer zone, filled by zeros, and therefore replacing the desired command does not require reformatting the entire file and can be performed even without using the arm64-to-sim utility at all with the help of a standard xcrun vtool instrument.

After running this utility, the load command should look like this:

Load command 35
      cmd LC_BUILD_VERSION
      cmdsize 24
      platform IOSSIMULATOR
      minos 8.0
      sdk 15.4
      ntools 0

Below you can see a basic script, that replaces fat frameworks, that support arm64 for devices (and maybe some other architectures), on frameworks that support only arm64 for simulators. arm64-to-sim utility is here used only for static object files, but, as it’s already been mentioned above, one could use it and for dynamic libs (in such a case one should pass one more argument — special flag, indicating that script should deal with dynamic library). In the example below, it’s intentionally not used for dynamic binaries, to show how we can use standard xcrun vtool instrument (as the original article does).

LIB_NAME=$1
MIN_IOS=${2:-"12.0"}
SDK=${3:-"13.0"}

lipo -thin arm64 ${LIB_NAME}.framework/${LIB_NAME} -output ${LIB_NAME}.arm64

if xcrun vtool -show ${LIB_NAME}.framework/${LIB_NAME}; then
# 7 - iOS Simulator platform
  xcrun vtool -arch arm64 -set-build-version 7 $MIN_IOS $SDK -replace -output ${LIB_NAME}.framework/${LIB_NAME} ${LIB_NAME}.arm64
  exit 0
fi

mkdir ${LIB_NAME}.arm64.objects
cd ${LIB_NAME}.arm64.objects
ar x ../${LIB_NAME}.arm64
for file in *.o; do arm64-to-sim $file $MIN_IOS $SDK; done;
ar crv ../${LIB_NAME}.arm64-sim *.o
cd ../

cp ${LIB_NAME}.arm64-sim ${LIB_NAME}.framework/${LIB_NAME}

What exactly does the above script do?

1. First of all, it extracts library for arm64 architecture from fat framework via command lipo -thin — exactly that library will be patched in the next steps (lipo tool is widely used for merging several libs for different architectures in one fat framework and vice-versa — thinning libs for concrete architectures from one fat framework). If the source framework is not fat and supports only arm64 architecture for real devices — this step might be skipped.
2. After that, script tries to read information about load commands via xcrun vtool utility (that works correctly only on Mach-O files and returns error on archives). On success, script replaces the command with the required one, explicitly specifying the platform code (7 for iOS Simulator platform), as well as the minos and sdk parameters. This completes the work of the script, because all changes have already been made to the final file in that step.
3. If the file was not Mach-O, that means for script that it’s dealing with a static framework that contains a static archive of object files. So, it needs to be unzipped to extract all of the object files.
4. Each object file is replaced using an arm64-to-sim tool, that not only updates the load command, but also, as it’s already been mentioned above, shifts the following contents of the file by the difference between the sizes of load commands. This is done for each object file in a loop.
5. After all of the object files were updated, the script repacks them into a static archive.
6. Finally, this archive replaces the original one in the given (thinned) framework from the first step.

As a successful result, script will generate a framework that supports only arm64 for simulators (without other architectures, that might be originally vendored by fat framework). But this procedure is already enough to build an app on arm64 simulator (of course, after all of the dependencies without its support will be replaced via a given script). The last thing left to say is that script works only for fat frameworks. If we are talking about a framework or a dynamic library for a single architecture — you don’t need to use the lipo command at all. If you need to update the static library (files with the .a extension) — it’s enough to perform the steps similar to p.p. 3–5.

XCFrameworks as a clue

In the previous section, it was shown how you can use the framework containing symbols for arm64 devices to generate framework for arm64 simulators to actually build the app for that architecture and platform. However, this procedure was performed very roughly, because it removed support for all other architectures, including arm64 itself for devices. Fat dependencies are usually used to merge several architectures together, but they don’t allow to support multiple implementations for the same architecture for different platforms at the same time. Here xcframework comes to the scene.

Despite the fact that several years have already passed since the announcement of this technology, the process of building xcframeworks is still not obvious and not easy enough to use. This mostly explains why not all developers still supported this format.

An approximate algorithm for creating xcframework is described below:

1. It is necessary to build frameworks (or libraries) for each supported platform and architecture. As a baseline, one should support three such versions:
   - arm64 for iOS devices;
   - arm64 for iOS Simulators;
   - x86_64 for iOS Simulators (because not everyone is still developing on M1, simulators for older versions (12/13) still need x86_64, and there might be different problems during the transfer of infrastructure, tests and other CI processes to arm64 architecture).
2. For each platform, it’s necessary to combine all the frameworks, collected in the previous step, together (into a fat framework via lipo -create command, if several architectures are supported for concrete platform). As a baseline, there are two platforms:
   - iOS devices (arm64) — only arm64 architecture is used in modern devices, so we don’t need do any additional actions;
   - iOS Simulators (arm64+x86_64).
3. Finally, you need to combine all the frameworks from step 2, separated by platforms, into a single xcframework using xcodebuild -create-xcframework command. That command will check that frameworks are correct, will generate Info.plist file with structure definition and will generate a well-formed xcframework.
4. For static libraries, the scheme is similar, only all actions with the lipo command must be performed with the library itself, and one should use -library flag instead of -framework and provide additionally paths to headers directories with special flag -headers, if needed, when using xcodebuild -create-xcframework command.

Specifically in our case study, the scheme above works fine, and generated xcframeworks are indeed well-formed. To confirm this, you can use script form the sample repository:

$ pod install
$ ./create-xcframeworks-with-arm64sim.sh Pods
$ find Pods -name ".xcframework"
Pods/libyuv-iOS/libyuv.xcframework
Pods/MNN/MNN.xcframework
Pods/MOBFoundation/MOBFoundation/MOBFoundation.xcframework
Pods/mob_sharesdk/ShareSDK/ShareSDK.xcframework
Pods/mob_sharesdk/ShareSDK/Support/Required/ShareSDKConnector.xcframework

However, I discovered some subtleties that need to be taken into account in cases where a dependency includes a swiftmodule. Despite the fact that swift dependencies are often distributed through the newfangled SPM (Swift Package Manager), in some cases, where there is no desire to share the source code, swift modules can also be distributed using binary frameworks.

Firstly (and this was announced at Binary Frameworks in Swift, WWDC’19 after xcframeworks announcement), the latest versions of swift support the mechanism of library evolution — it allows to add special flags to achieve such a behavior, that already assembled dependencies on swift do not need to be rebuilt when the language version changes. By default, when creating a framework on swift, the file structure looks like this:

$ tree SimpleFramework.framework
SimpleFramework.framework
├── Headers
│   ├── SimpleFramework-Swift.h
│   └── SimpleFramework.h
├── Info.plist
├── Modules
│   ├── SimpleFramework.swiftmodule
│   │   ├── Project
│   │   │   └── arm64-apple-ios-simulator.swiftsourceinfo
│   │   ├── arm64-apple-ios-simulator.abi.json
│   │   ├── arm64-apple-ios-simulator.swiftdoc
│   │   └── arm64-apple-ios-simulator.swiftmodule
│   └── module.modulemap
├── SimpleFramework
└── _CodeSignature
    └── CodeResources

If one specifies the BUILD_LIBRARY_FOR_DISTRIBUTION=”YES” flag during the build, the structure will become like this:

$ tree SimpleFramework.framework
SimpleFramework.framework
├── Headers
│   ├── SimpleFramework-Swift.h
│   └── SimpleFramework.h
├── Info.plist
├── Modules
│   ├── SimpleFramework.swiftmodule
│   │   ├── Project
│   │   │   └── arm64-apple-ios-simulator.swiftsourceinfo
│   │   ├── arm64-apple-ios-simulator.abi.json
│   │   ├── arm64-apple-ios-simulator.private.swiftinterface
│   │   ├── arm64-apple-ios-simulator.swiftdoc
│   │   ├── arm64-apple-ios-simulator.swiftinterface
│   │   └── arm64-apple-ios-simulator.swiftmodule
│   └── module.modulemap
├── SimpleFramework
└── _CodeSignature
    └── CodeResources

The key difference lies in the .swiftinterface files, which describe the general public interface of swift modules, that is resistant to changes in language versions. abi.json and private.swiftinterface files appeared in the latest version of Xcode (Xcode 14) — Xcode 13.4.1 did not create such files.

Look at the sample source swift file and generated .swiftinterface file for it:

import Foundation

public class Simple {
    var simpleVar: Int

    public init() {
        simpleVar = 42
    }

    public static let simpleConstant = 42
    static let simplePrivateConstant = 10

    public class func simpleClassFunc() -> Int {
        simpleLogic(for: simpleConstant, base: 0)
    }

    public func simpleFunc(for base: Int) -> Int {
        Simple.simpleLogic(for: simpleVar, base: base)
    }

    private class func simpleLogic(for num: Int, base: Int) -> Int {
        simplePrivateConstant * num
    }
}

// swift-interface-format-version: 1.0
// swift-compiler-version: Apple Swift version 5.7 (swiftlang-5.7.0.127.4 clang-1400.0.29.50)
// swift-module-flags: -target arm64-apple-ios16.0-simulator -enable-objc-interop -enable-library-evolution -swift-version 5 -enforce-exclusivity=checked -Onone -module-name SimpleFramework
// swift-module-flags-ignorable: -enable-bare-slash-regex
 
import Foundation
@_exported import SimpleFramework
import Swift
import _Concurrency
import _StringProcessing
 
public class Simple {
  public init()
  public static let simpleConstant: Swift.Int
  public class func simpleClassFunc() -> Swift.Int
  public func simpleFunc(for base: Swift.Int) -> Swift.Int
  @objc deinit
}

You can also specify additional flags: CODE_SIGN_IDENTITY=”” CODE_SIGNING_REQUIRED=”NO” CODE_SIGNING_ALLOWED=”NO”, in order exclude signature information from the framework itself, transferring responsibility for the signature to the developer, that will use this framework as an external dependency — in such a case, the _CodeSignature directory will disappear from the file structure.

Finally, official documentation about Library Evolution in Swift says that adding the BUILD_LIBRARY_FOR_DISTRIBUTION build option generates the so-called dispatch thunk function for every protocol requirement. So, if your project is built with that flag and uses some source-dependencies (not binary frameworks, but source — like this one, for example), so it’s also needed to be rebuilt with that flag, that needs to update .podspec file directly, adding appropriate project build option there. If you don’t do that, imagine if someone will use in separate project X your built for distribution binary [xc-]framework Y, it will also be transitively dependent on the framework’s own source-dependency Z. In such a case, framework Y will expect dispatch thunk symbols from source-dependency Z, but when you will build this separate project X, source dependency Z will be built without BUILD_LIBRARY_FOR_DISTRIBUTION option, that will lead to linkage error. So, any project X will have to add that option for its implicit (sic!) dependency Z manually, that’s break encapsulation of a framework Y. That’s why it’s the responsibility of Y to provide such a patched .podspec file for Z, that will contain BUILD_LIBRARY_FOR_DISTRIBUTION option for auto-generating Z-target project — so framework Y and any project X will use identical options to build source-dependency Z, and there will be no linkage issues. To do that — just add something like that to .podspec file of source-dependency Z and distribute it to end-users alongside with .podspec file for the framework Y (or just ask the maintainer to add it in the source repo):

spec.pod_target_xcconfig = {
  'BUILD_LIBRARY_FOR_DISTRIBUTION' => 'YES',
}

The second nuance is related to the process of creating a fat framework from the frameworks with swiftmodules. The problem is that unlike resources, which usually do not differ for simulators and devices, the files inside swiftmodules differ for each of the architectures and platforms. Therefore, in addition to updating the binary file of the framework itself via lipo -create command, it’s necessary to merge the contents of swiftmodules in the final framework. For example, this could be done using the rsync -avh command for the Modules subdirectory.

However, in addition to these nuances, there is a third, the most serious one. The fact is that when creating a xcframework from valid fat frameworks, the .swiftmodule directories for all architectures are lost. This is shown in the listing below:

$ mkdir tmp-xcframework
$ xcodebuild -project SimpleFramework.xcodeproj -scheme SimpleFramework -configuration Release -derivedDataPath ./DerivedData -sdk 'iphoneos' -arch arm64 -parallelizeTargets -jobs 8 CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED="NO" CODE_SIGNING_ALLOWED="NO" BUILD_LIBRARY_FOR_DISTRIBUTION="YES" clean build
$ tree ./DerivedData/Build/Products/Release-iphoneos/SimpleFramework.framework
./DerivedData/Build/Products/Release-iphoneos/SimpleFramework.framework
├── Headers
│   ├── SimpleFramework-Swift.h
│   └── SimpleFramework.h
├── Info.plist
├── Modules
│   ├── SimpleFramework.swiftmodule
│   │   ├── Project
│   │   │   └── arm64-apple-ios.swiftsourceinfo
│   │   ├── arm64-apple-ios.abi.json
│   │   ├── arm64-apple-ios.private.swiftinterface
│   │   ├── arm64-apple-ios.swiftdoc
│   │   ├── arm64-apple-ios.swiftinterface
│   │   └── arm64-apple-ios.swiftmodule
│   └── module.modulemap
└── SimpleFramework

$ mkdir tmp-xcframework/device
$ cp -a ./DerivedData/Build/Products/Release-iphoneos/SimpleFramework.framework tmp-xcframework/device
$ xcodebuild -project SimpleFramework.xcodeproj -scheme SimpleFramework -configuration Debug -derivedDataPath ./DerivedData -sdk 'iphonesimulator' -arch x86_64 -parallelizeTargets -jobs 8 CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED="NO" CODE_SIGNING_ALLOWED="NO" BUILD_LIBRARY_FOR_DISTRIBUTION="YES" clean build
$ tree ./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework
./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework
├── Headers
│   ├── SimpleFramework-Swift.h
│   └── SimpleFramework.h
├── Info.plist
├── Modules
│   ├── SimpleFramework.swiftmodule
│   │   ├── Project
│   │   │   └── x86_64-apple-ios-simulator.swiftsourceinfo
│   │   ├── x86_64-apple-ios-simulator.abi.json
│   │   ├── x86_64-apple-ios-simulator.private.swiftinterface
│   │   ├── x86_64-apple-ios-simulator.swiftdoc
│   │   ├── x86_64-apple-ios-simulator.swiftinterface
│   │   └── x86_64-apple-ios-simulator.swiftmodule
│   └── module.modulemap
└── SimpleFramework

$ mkdir tmp-xcframework/x86_64-sim
$ cp -a ./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework tmp-xcframework/x86_64-sim
$ xcodebuild -project SimpleFramework.xcodeproj -scheme SimpleFramework -configuration Debug -derivedDataPath ./DerivedData -sdk 'iphonesimulator' -arch arm64 -parallelizeTargets -jobs 8 CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED="NO" CODE_SIGNING_ALLOWED="NO" BUILD_LIBRARY_FOR_DISTRIBUTION="YES" clean build
$ tree ./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework
./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework
├── Headers
│   ├── SimpleFramework-Swift.h
│   └── SimpleFramework.h
├── Info.plist
├── Modules
│   ├── SimpleFramework.swiftmodule
│   │   ├── Project
│   │   │ └── arm64-apple-ios-simulator.swiftsourceinfo
│   │   ├── arm64-apple-ios-simulator.abi.json
│   │   ├── arm64-apple-ios-simulator.private.swiftinterface
│   │   ├── arm64-apple-ios-simulator.swiftdoc
│   │   ├── arm64-apple-ios-simulator.swiftinterface
│   │   └── arm64-apple-ios-simulator.swiftmodule
│   └── module.modulemap
└── SimpleFramework

$ mkdir tmp-xcframework/arm64-sim
$ cp -a ./DerivedData/Build/Products/Debug-iphonesimulator/SimpleFramework.framework tmp-xcframework/arm64-sim
$ mkdir tmp-xcframework/sim
$ cp -a tmp-xcframework/arm64-sim/SimpleFramework.framework tmp-xcframework/sim
$ lipo -create tmp-xcframework/x86_64-sim/SimpleFramework.framework/SimpleFramework tmp-xcframework/arm64-sim/SimpleFramework.framework/SimpleFramework -output tmp-xcframework/sim/SimpleFramework.framework/SimpleFramework
$ tree tmp-xcframework/sim
tmp-xcframework/sim
└── SimpleFramework.framework
    ├── Headers
    │   ├── SimpleFramework-Swift.h
    │   └── SimpleFramework.h
    ├── Info.plist
    ├── Modules
    │   ├── SimpleFramework.swiftmodule
    │   │   ├── Project
    │   │   │   └── arm64-apple-ios-simulator.swiftsourceinfo
    │   │   ├── arm64-apple-ios-simulator.abi.json
    │   │   ├── arm64-apple-ios-simulator.private.swiftinterface
    │   │   ├── arm64-apple-ios-simulator.swiftdoc
    │   │   ├── arm64-apple-ios-simulator.swiftinterface
    │   │   └── arm64-apple-ios-simulator.swiftmodule
    │   └── module.modulemap
    └── SimpleFramework

$ rsync -avh tmp-xcframework/x86_64-sim/SimpleFramework.framework/Modules tmp-xcframework/sim/SimpleFramework.framework
$ tree tmp-xcframework/sim
tmp-xcframework/sim
└── SimpleFramework.framework
    ├── Headers
    │   ├── SimpleFramework-Swift.h
    │   └── SimpleFramework.h
    ├── Info.plist
    ├── Modules
    │   ├── SimpleFramework.swiftmodule
    │   │   ├── Project
    │   │   │   ├── arm64-apple-ios-simulator.swiftsourceinfo
    │   │   │   └── x86_64-apple-ios-simulator.swiftsourceinfo
    │   │   ├── arm64-apple-ios-simulator.abi.json
    │   │   ├── arm64-apple-ios-simulator.private.swiftinterface
    │   │   ├── arm64-apple-ios-simulator.swiftdoc
    │   │   ├── arm64-apple-ios-simulator.swiftinterface
    │   │   ├── arm64-apple-ios-simulator.swiftmodule
    │   │   ├── x86_64-apple-ios-simulator.abi.json
    │   │   ├── x86_64-apple-ios-simulator.private.swiftinterface
    │   │   ├── x86_64-apple-ios-simulator.swiftdoc
    │   │   ├── x86_64-apple-ios-simulator.swiftinterface
    │   │   └── x86_64-apple-ios-simulator.swiftmodule
    │   └── module.modulemap
    └── SimpleFramework

$ xcodebuild -create-xcframework -framework tmp-xcframework/device/SimpleFramework.framework -framework tmp-xcframework/sim/SimpleFramework.framework -output tmp-xcframework/SimpleFramework.xcframework
$ tree ./tmp-xcframework/SimpleFramework.xcframework
./tmp-xcframework/SimpleFramework.xcframework
├── Info.plist
├── ios-arm64
│   └── SimpleFramework.framework
│       ├── Headers
│       │ ├── SimpleFramework-Swift.h
│       │ └── SimpleFramework.h
│       ├── Info.plist
│       ├── Modules
│       │   ├── SimpleFramework.swiftmodule
│       │   │   ├── Project
│       │   │   │   └── arm64-apple-ios.swiftsourceinfo
│       │   │   ├── arm64-apple-ios.abi.json
│       │   │   ├── arm64-apple-ios.private.swiftinterface
│       │   │   ├── arm64-apple-ios.swiftdoc
│       │   │   └── arm64-apple-ios.swiftinterface
│       │   │       #.swiftmodule is absent
│       │   └── module.modulemap
│       └── SimpleFramework
└── ios-arm64_x86_64-simulator
    └── SimpleFramework.framework
        ├── Headers
        │   ├── SimpleFramework-Swift.h
        │   └── SimpleFramework.h
        ├── Info.plist
        ├── Modules
        │   ├── SimpleFramework.swiftmodule
        │   │   ├── Project
        │   │   │   ├── arm64-apple-ios-simulator.swiftsourceinfo
        │   │   │   └── x86_64-apple-ios-simulator.swiftsourceinfo
        │   │   ├── arm64-apple-ios-simulator.abi.json
        │   │   ├── arm64-apple-ios-simulator.private.swiftinterface
        │   │   ├── arm64-apple-ios-simulator.swiftdoc
        │   │   ├── arm64-apple-ios-simulator.swiftinterface
        │   │   ├── x86_64-apple-ios-simulator.abi.json
        │   │   ├── x86_64-apple-ios-simulator.private.swiftinterface
        │   │   ├── x86_64-apple-ios-simulator.swiftdoc
        │   │   └── x86_64-apple-ios-simulator.swiftinterface
        │   │       # .swiftmodule are absent
        │   └── module.modulemap
        └── SimpleFramework

Actually, there is a radar that has still been open since 2019. This leads to build failure, because the build system can’t find any symbols in the imported module.

The solution to the latter problem is quite simple and straightforward — after creating xcframework, you need to copy the .swiftmodule files separately for each architecture:

$ cp -a tmp-xcframework/sim/SimpleFramework.framework/Modules/SimpleFramework.swiftmodule/*.swiftmodule tmp-xcframework/SimpleFramework.xcframework/ios-*simulator/SimpleFramework.framework/Modules/SimpleFramework.swiftmodule
$ cp -a tmp-xcframework/device/SimpleFramework.framework/Modules/SimpleFramework.swiftmodule/*.swiftmodule tmp-xcframework/SimpleFramework.xcframework/ios-arm64/SimpleFramework.framework/Modules/SimpleFramework.swiftmodule
$ tree ./tmp-xcframework/SimpleFramework.xcframework
./tmp-xcframework/SimpleFramework.xcframework
├── Info.plist
├── ios-arm64
│   └── SimpleFramework.framework
│       ├── Headers
│       │   ├── SimpleFramework-Swift.h
│       │   └── SimpleFramework.h
│       ├── Info.plist
│       ├── Modules
│       │   ├── SimpleFramework.swiftmodule
│       │   │   ├── Project
│       │   │   │   └── arm64-apple-ios.swiftsourceinfo
│       │   │   ├── arm64-apple-ios.abi.json
│       │   │   ├── arm64-apple-ios.private.swiftinterface
│       │   │   ├── arm64-apple-ios.swiftdoc
│       │   │   ├── arm64-apple-ios.swiftinterface
│       │   │   └── arm64-apple-ios.swiftmodule
│       │   └── module.modulemap
│       └── SimpleFramework
└── ios-arm64_x86_64-simulator
    └── SimpleFramework.framework
        ├── Headers
        │   ├── SimpleFramework-Swift.h
        │   └── SimpleFramework.h
        ├── Info.plist
        ├── Modules
        │   ├── SimpleFramework.swiftmodule
        │   │   ├── Project
        │   │   │   ├── arm64-apple-ios-simulator.swiftsourceinfo
        │   │   │   └── x86_64-apple-ios-simulator.swiftsourceinfo
        │   │   ├── arm64-apple-ios-simulator.abi.json
        │   │   ├── arm64-apple-ios-simulator.private.swiftinterface
        │   │   ├── arm64-apple-ios-simulator.swiftdoc
        │   │   ├── arm64-apple-ios-simulator.swiftinterface
        │   │   ├── arm64-apple-ios-simulator.swiftmodule
        │   │   ├── x86_64-apple-ios-simulator.abi.json
        │   │   ├── x86_64-apple-ios-simulator.private.swiftinterface
        │   │   ├── x86_64-apple-ios-simulator.swiftdoc
        │   │   ├── x86_64-apple-ios-simulator.swiftinterface
        │   │   └── x86_64-apple-ios-simulator.swiftmodule
        │   └── module.modulemap
        └── SimpleFramework

Although all is looking well now for Xcode 13.*, Xcode 14 brought another pitfall, related to the public SimpleFramework-Swift.h header. If you look at SimpleFramework.xcframework/ios-arm64_x86_64-simulator/SimpleFramework.framework/Headers/SimpleFramework-Swift.h (that is swift header for simulator fat framework), you should see some newly added ifdef directives for each architecture:

#if 0
#elif defined(__arm64__) && __arm64__
// Generated by Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
#ifndef SIMPLEFRAMEWORK_SWIFT_H
#define SIMPLEFRAMEWORK_SWIFT_H
// …
#elif defined(__x86_64__) && __x86_64__
// Generated by Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
#ifndef SIMPLEFRAMEWORK_SWIFT_H
#define SIMPLEFRAMEWORK_SWIFT_H
// …
#else
#error unsupported Swift architecture
#endif

But if you use the scheme above, you will get only one ifdef branch at all. So, it will cause the build to fail on one of target architectures. It’s caused with the way we generated fat simulator framework:

$ cp -a tmp-xcframework/arm64-sim/SimpleFramework.framework tmp-xcframework/sim

That command just copied the overall framework structure, and after that we patched its binary. But since Xcode 14, headers are not identical for different architectures, so it’s needed to merge them correctly somehow. Actually, the simplest way to do it is to build fat-framework. To do that, one should just replace xcodebuild commands for each architecture like this:

$ xcodebuild -project SimpleFramework.xcodeproj -scheme SimpleFramework -configuration Debug -derivedDataPath ./DerivedData -sdk ‘iphonesimulator’ -arch x86_64 -parallelizeTargets -jobs 8 CODE_SIGN_IDENTITY=”” CODE_SIGNING_REQUIRED=”NO” CODE_SIGNING_ALLOWED=”NO” BUILD_LIBRARY_FOR_DISTRIBUTION=”YES” clean build

to something like that:

$ xcodebuild -project SimpleFramework.xcodeproj -scheme SimpleFramework -configuration Debug -derivedDataPath ./DerivedData -sdk ‘iphonesimulator’ -parallelizeTargets -jobs 8 CODE_SIGN_IDENTITY=”” CODE_SIGNING_REQUIRED=”NO” CODE_SIGNING_ALLOWED=”NO” BUILD_LIBRARY_FOR_DISTRIBUTION=”YES” ONLY_CURRENT_ARCH=”NO” clean build

After that you will receive the final and ready to be integrated to the final xcframework — fat framework for simulators. That pitfall is related only to public *-Swift.h files, so if it’s absent, no problems with the overall scheme above should be met.

Finally, the contents of the auto-generated Info.plist, that defines the internal structure of the formed xcframework, is shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>AvailableLibraries</key>
    <array>
        <dict>
            <key>LibraryIdentifier</key>
            <string>ios-arm64</string>
            <key>LibraryPath</key>
            <string>SimpleFramework.framework</string>
            <key>SupportedArchitectures</key>
            <array>
                <string>arm64</string>
            </array>
            <key>SupportedPlatform</key>
            <string>ios</string>
        </dict>
        <dict>
            <key>LibraryIdentifier</key>
            <string>ios-arm64_x86_64-simulator</string>
            <key>LibraryPath</key>
            <string>SimpleFramework.framework</string>
            <key>SupportedArchitectures</key>
            <array>
                <string>arm64</string>
                <string>x86_64</string>
            </array>
            <key>SupportedPlatform</key>
            <string>ios</string>
            <key>SupportedPlatformVariant</key>
            <string>simulator</string>
        </dict>
    </array>
    <key>CFBundlePackageType</key>
    <string>XFWK</string>
    <key>XCFrameworkFormatVersion</key>
    <string>1.0</string>
</dict>
</plist>

Some useful notes about known pitfalls, when moving to XCFrameworks

It’s not enough just to create xcframework — it still needs to be properly integrated into the project. When we use CocoaPods, the pods installation system extracts type of dependency from .podspec file and automatically configures its integration lifecycle, adding additional build phases. So, from the developer’s viewpoint, one should provide enough information to CocoaPods to properly integrate dependency into the project.

It might be not trivial and obvious enough for first time, so you can follow the following steps below:

1. You need to assemble xcframework itself and all additional assets (like images or other resources, if exist and not already bundled into frameworks) into single zip archive. There are some situations, when some pods provide resources bundles separately from xcframework or several related xcframeworks together in one pod.
2. The resulting zip archive should be uploaded to some storage or repository. For instance, it could be company-wide s3 storage (you might use s3 minio server, for instance, or something else), but for this paper I used the same repository on GitHub for such purposes, where a sample project was located.
3. It’s necessary to modify original .podspec file (you might find it in ~/Library/Caches/CocoaPods/Pods/Specs/Release/ directory after you have installed them at least once) — you should change path to framework to the path to xcframework for vendored_frameworks key, also you should change source URL to uploaded archive from step 2, pod’s version, and provide paths to additional resources, headers etc. Due to the fact that resources bundles usually are similar for simulator and device versions of the app (from ios-arm64 directory of xcframework), one should always use device-versions as in the example below to avoid their duplication in the final project.
4. Finally, for each pod you need to specify a path to the new .podspec file in the Podfile.

Take a look at the example below for one of patched dependencies for sample demo project — you can find them all and the final project structure, using xcframeworks, as well, in the xcframeworks branch (feel free to change description of pod — here it intentionally remained unchanged to simplify diff finding).

{
    "name": "libyuv-iOS",
    "version": "1.0.2-xcframework",
    "license": "BSD",
    "summary": "libyuv arm7 & arm64 library for iOS",
    "homepage": "https://chromium.googlesource.com/libyuv/libyuv/",
    "authors": "The LibYuv Project Authors",
    "source": {
        "http": "https://github.com/leshiy1295/xcframeworks-with-arm64sim-demo/blob/xcframeworks/xcframeworks-storage/libyuv-iOS/1.0.2-xcframework/libyuv-iOS.xcframework.zip?raw=true"
    },
    "platforms": {
        "ios": "8.0"
    },
    "description": "libyuv is an open source project that includes YUV scaling and conversion functionality.",
    "source_files": [
        "libyuv.xcframework/ios-arm64/**/headers/**/*.h"
    ],
    "vendored_frameworks": "libyuv.xcframework",
    "requires_arc": false
}

It is worth noting that such a solution with explicit storing .podspec files inside the repository should be considered as temporary, just for a transition period. After all works with moving to xcframeworks are done, one should push these .podspec files to the separate repository (if public one is not suitable for your case due to any purpose) — in such a situation you should directly specify the URL of this repository with source command and remove :podspec properties for each pod in Podfile — so their search will be carried out automatically in the remote source repository.

There is also one little common pitfall here — if something goes wrong, first of all check CocoaPods caches for the .podspec file and pod itself. They live in ~/Library/Caches/CocoaPods/Pods directory — in [Specs/]Release subfolder, when using concrete version in Podfile, and in [Specs/]External, when using path to podspec. The common recommendation — if you change something with your pod — change its version too, because all versions are cached locally, and it may become painful, if you could already try to use them on CI builds for your in-progress merge request.

Are there some other difficulties you can face? Well, in app, given for audit, I turned out that some pods contained Intel-architecture-dependent code — that’s why I added specific subset of dependencies only for arm64 architecture to provide missing symbols for that architecture and to avoid the duplication of such symbols on the linking build stage for x86_64 architecture. To make such exclusions, it’s enough to do a simple trick in the Podfile by implementing some additional logic in post_install hook like the following:

1. Remove from generating xcconfig files (they have the format key = value and could be configured based on the information from .podspec files at key with appropriate name) all substrings for the key OTHER_LDFLAGS (which defines additional flags used at the linking stage), related to the special frameworks, that looks like: -framework SOME_FRAMEWORK;
2. After that, add the same line, but targeted for certain platforms and architectures. In my case, for iOS Simulators on arm64, it was the following option:
   OTHER_LDFLAGS[sdk=iphonesimulator*][arch=arm64].
   You can read more about such methods of targeting options, for, example, here.

Another detail is the set of supported architectures. Throughout the article, I suggested using only arm64 for devices, and x86_64 with arm64 for simulators (and also the main patch script uses that assumption). However, for too old models, it might still be necessary to use other architectures, such as armv7 or i386 (like in FirebaseAnalytics.xcframework). To do that, you should follow the same way I described for x86_64 support for simulators — you need to extract appropriate architecture from the original fat framework using lipo -thin command, and after completing all the deals with patching — add them back with the lipo -create command to the binaries. Keep in mind, that since Xcode 14, when dealing with *-Swift.h files, you should initially generate fat-framework to get the correct header. At the same moment, you can find in Xcode 14 Release Notes about deprecation of some of such architectures:

Building iOS projects with deployment targets for the armv7, armv7s, and i386 architectures is no longer supported. (92831716)

Also, when you have a deal with your own source code, and are going to provide xcframeworks with dSYM files — one should include these dSYM files into xcframeworks as well. Firstly, to generate such dSYM files don’t forget to add DEBUG_INFORMATION_FORMAT=dwarf-with-dsym to xcodebuild command or to the Xcode project settings (Note: if you take deal with pure swiftmodules, there will be no dSYM generated — all debug information will be automatically extracted from such swiftmodules and placed to the final app dSYM file, so there is no need to do something else on your side). Moreover, it’s not so obvious, but dSYM for fat frameworks also are prepared by merging dSYM for separate frameworks with the same lipo -create tool. After all the dSYMs are prepared, you should add -debug-symbols key, when creating xcframework, providing absolute (sic!) paths to dSYMs after the framework for each architecture like that (don’t forget to copy dSYM files to appropriate platform directories, as shown below):

lipo -create \
  $(pwd)/tmp-xcframework/x86_64-sim/SimpleFramework.framework.dSYM/Contents/Resources/DWARF/SimpleFramework \
  $(pwd)/tmp-xcframework/arm64-sim/SimpleFramework.framework.dSYM/Contents/Resources/DWARF/SimpleFramework \
  -output $(pwd)/tmp-xcframework/sim/SimpleFramework.framework.dSYM/Contents/Resources/DWARF/SimpleFramework

xcodebuild -create-xcframework \
  -framework tmp-xcframework/device/SimpleFramework.framework \
  -debug-symbols $(pwd)/tmp-xcframework/device/SimpleFramework.framework.dSYM \
  -framework tmp-xcframework/sim/SimpleFramework.framework \
  -debug-symbols $(pwd)/tmp-xcframeworks/sim/SimpleFramework.framework.dSYM \
  -output tmp-xcframework/SimpleFramework.xcframework

In Xcode 14 beta there were certain problems with violating the principles of library evolution even when setting all required options during the build. There was rather active discussion on the Apple forum. I have faced once with such a situation, when all worked well with Xcode 13.4.1, but not with stable version of Xcode 14 — the problem was somehow related to actors model, introduced in swift, and was caused with using UI-main-thread-specific API as default arguments values in public API functions (concrete error message became look like “Main actor-isolated property ‘bounds’ can not be referenced from a non-isolated context“ in Xcode 14.2 and was uninformative in previous Xcode 14 versions, looking like this:

“Failed to build module ***; this SDK is not supported by the compiler (the SDK is built with ‘Apple Swift version 5.6.1 (swiftlang-5.6.0.323.66 clang-1316.0.20.12)’, while this compiler is ‘Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)’). Please select a toolchain which matches the SDK.”).

When I hid the use of this API inside the body of that function, the problem disappeared. After that fix, for the last year, I have not found any serious problems related to the library evolution mechanism.

Also, one of promising and growing technologies KMM (Kotlin Multiplatform Mobile) is actively used nowadays to develop a cross-platform client, sharing code (for instance, for working with Backend Driven UI or other business logic) between all supported platforms. Tutorials from the web, unfortunately, become obsolete very quickly — IDE support is rapidly keeping up with the time. But there is a rather stable mechanism for the last several versions, that adds the opportunity to build a complete xcframework with correct structure and for all needed platforms via gradle or even on project initial configuration, marking appropriate options in configuration master. You can read more about that in official documentation. Also, on that page you might find information for building fat frameworks.

Despite the fact that the common scheme is briefly described there, there could be some troubles with iosSimulatorArm64 target support in the project or in some of its external dependencies. As an example of such support, you might look at pull request to one of open-source projects. Similar changes may be required for other build stages — for example, if you use KSP (Kotlin Symbol Processing), or even you might need to add some actual implementation for such a target, similarly to existing ones. After all of these preparations, you can follow documentation about xcframework creation according to the link above. Nevertheless, it’s easy enough to imagine one disadvantage of such an approach — you can assemble completely debug or release xcframework, but sometimes one would like to have debug framework for simulators and release — for devices inside one xcframework. You still can try to do that, calling gradle subtasks, but that becomes inconvenient. In such situations, you still can build distinct frameworks with needed configurations via gradle and then assemble them manually with approaches that were described in this paper earlier.

Finally, I was “lucky” to catch another strange behavior when linking on different operating systems — there was no problem on macOS Ventura beta versions, when build failed on previous OS version and Xcode 13.4.1). Bug was related to the unaligned pointers existence for arm64 architecture:

ld: unaligned pointer(s) for architecture arm64

The same problem appeared once also in the Firebase repository, but in my case it appeared with one of private legacy external dependencies, and I also did not have any access to its source code. The root of the problem was caused by using protobuf inside one of the modules via nanopb lib — code generation for C-structures used structures packing technique to lowerize final structures size, but that led to unaligned pointers appearance, that’s not supported on arm64 architecture, as we can see from the error message.

To fix this, I had to reconstruct the original .proto-file, based on existing public information from header-files about definitions of all structures. After that, one needed to regenerate .c files with disabled precompile-option of structure packing (it was usual define in nanopb lib). Finally, one needed to get object files via usual compiler, but (sic!), I had to compile not for macOS, but for iOS and for each of required architectures. After that, I’ve found appropriate object files from patched xcframeworks and replaced them with recompiled ones, then packed them again to libraries archives and put them into xcframework. So, initial problem was actually solved.

Despite the complexity of the procedure described above, this is quite a rare case, so I decided not to include a detailed example in the sample demo repository with such an example. Nevertheless, it’s useful to know that, if desired, you can overcome similar difficulties!

Life on arm64 iOS Simulators after moving to XCFrameworks

After I’ve integrated arm64 support for iOS Simulators, app hangs on start completely disappeared, i.e. the original problem was actually solved! Moreover, it turned out, that other side effects, that I encountered on M1, also disappeared — for example, swipes in the application started working without freezes and stops, when finger was taken off the screen, also pasteboard started to work fine (which stopped working on iOS 14+ on simulators). The latter is especially significant, because there were similar questions on forums on the web, but it was still not obvious at all that the reason was implicitly caused behind the scenes of the application process virtualization via Rosetta.

However, the celebration was short-lived. It turned out that when the project moved to arm64, we were faced with the fact that the p and po commands did not work when debugging, even where they used to work on x86_64 without any problems (on older versions of Xcode or on the new beta). In fact, I faced with two different behaviors, at least when debugging swift code:

1. Xcode 13.4.1 shown the following message:
   Couldn’t realize type of self.
2. Debugger on Xcode 14 just crashed all the time.

Nevertheless, it turned out that these problems can also be dealt with. The solution to the first problem was suggested by Apple itself at the WWDC’22 Debug Swift debugging with LLDB session. In that session, developers from Apple talked about the features of swift debugging. It turned out that for static libraries on swift (and app, provided for audit, used exactly this method of dependencies linkage), the debugger did not have enough information from dSYM to process interactive commands during debugging — moreover, the left and right panels of the debugger window in Xcode work differently and use separate sources of information.

Proof of Concept, dedicated to solving exactly this problem for one framework, can be found here. For a more systematic solution of this problem, it’s enough to add additional options for linking (the same OTHER_LDFLAGS setting key) for all projects in the Podfile in the post_install hook that is already familiar to us. The easiest way to do this is to bypass all dependencies and add such an option for each of them. As a result, approximately the following code should be sufficient:

post_install do |installer|
  def add_lldb_ast_for_swift_modules(build_settings, installer)
    ld_flags_key = 'OTHER_LDFLAGS'
    build_settings[ld_flags_key] ||= "$(inherited)"
    build_settings[ld_flags_key].strip!
    installer.generated_projects.each do |project|
      project.targets.each do |target|
        # adding explicit paths even if some of them do not exist - wildcards not working here…
        build_settings[ld_flags_key] += " -Wl,-add_ast_path,$(TARGET_BUILD_DIR)/#{target}/#{target}.framework/Modules/#{target}.swiftmodule/$(NATIVE_ARCH_ACTUAL)-apple-$(SHALLOW_BUNDLE_TRIPLE).swiftmodule"
      end
    end
  end

  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      xcconfig_path = config.base_configuration_reference.real_path
      build_settings = Hash[*File.read(xcconfig_path).lines.map{|x| x.split(/\s+=\s+/, 2)}.flatten]
      add_lldb_ast_for_swift_modules(build_settings, installer) if config.name == "Debug"
    end
  end
end

With the second problem, everything turned out to be somewhat more interesting — for some reason, the debugger tried to use iphoneos SDK instead of iphonesimulator, even when the app was assembled for the iOS simulator only and it was clear from the build logs, that the correct version of SDK used for build. I didn’t see this particular problem on the web, but on forums, discussing lldb errors, Apple experts confirmed that sometimes an SDK for macOS was erroneously set up for simulators.

Anyway, in my case, I managed to solve this problem by adding a LLDBInitFile file, where one could explicitly set the path to the correct SDK for the debugger. Since I didn’t want to hardcode that path, which also depends on the SDK version and target platform, but can be easily obtained from ENV variables during the build process, I implemented the following:

1. I added a separate build phase in the form of a script that updates the contents of the LLDBInitFile file using the following command:
   echo “settings set target.sdk-path ${SDKROOT}” > “${SRCROOT}/LLDBInitFile”
2. In the application build scheme, the path to the LLDBInitFile being created was explicitly specified (the same as was suggested by default).
3. Finally, I added LLDBInitFile to .gitignore, because it’s generated automatically and may change depending on the build to a real device or simulator.

As a result of performing all the described procedures, the debugger stopped crashing, and life became good!

However, in some cases, already in the released versions of Xcode 14, sometimes the debugger still refuses to work for unknown reasons –perhaps Apple will fix it in the future. For example, in some cases it still gives the error “cannot find ‘$__lldb_injected_self’ in scope“, familiar to the community for 7 years or more, in some cases does not output the values of the fields of CG structures (the only found workaround is to use “Print description” option from the left debugger window from the variable context menu), and sometimes it even “does not see” swift structures on the current stack frame, that are passed as function arguments. In the latter case, it’s especially interesting that the same structures are printed without any problems on the previous stack frame (at calling function), and when structs are replaced with classes.

What exactly is the reason of such strange and buggy behaviors of the lldb debugger — lazy copy-on-write on structures copying in swift, other hidden optimizations (that, of course, were disabled via project settings with flag -O0 when I reproduced such problems) or something else — we can only guess. Perhaps, this is still due to the relative novelty of the M1 platform as such and the insufficient number of projects actively using it in development without Rosetta (however, Xcode 14.3 disables even this opportunity). I hope that this article will allow more developers to switch to M1 and arm64 architecture using xcframeworks, and will contribute to the development of these technologies and reduction of such “oddities” and bugs! If you have ideas (or even better — stable solutions) that allow you to overcome these oddities — feel free to share them in the comments to this paper!

P.S.

I understand that the material in this article is quite complex, because it touches on a lot of topics that are usually not observed in different iOS development courses, and they practically do not occur in product development. Therefore, I would like to leave an additional list of sources that turned out to be useful to me personally when I was diving into this area.

About Rosetta2:

• How does Rosetta 2 work?
• Running Intel Code on your M1 Mac: Rosetta 2 and OAH
• Project Champollion

About patching object files from arm64 for devices to arm64 for simulators:

• Hacking native ARM64 binaries to run on the iOS Simulator
• Hacking native ARM64 binaries to run on the iOS Simulator — the dynamic framework edition

Basic step-by-step tutorial videos about creating libraries and xcframeworks:

• Creating and Publishing Custom iOS Framework using Cocoapods | Swift 5, XCode 11
• How to Create a CocoaPods Library to share code
• How to Create an Umbrella Framework
• How to Create an XCFramework — Frameworks and Static Libraries
• How to build and XCFramework for Apple Silicon M1 and Intel macs

Advanced materials about CocoaPods, libraries, frameworks and xcframeworks:

• CocoaPods Under The Hood
• Pod Authors Guide to CocoaPods Frameworks
• Binary Frameworks in Swift, WWDC’19
• Creating a multiplatform binary framework bundle
• Swift Libraries, frameworks, swift package. What’s the difference?
• It Looks Like You Are Trying to Use a Framework
• Embedding Frameworks in An App
• Frameworks: embed or not embed that’s the question
• iOS: Umbrella Framework with CocoaPods
• Introduction to static and dynamic, libraries and frameworks on iOS (and macOS)
• Advanced Static vs Dynamic libraries and frameworks on iOS (and macOS)
• S03E07: Differences in Dynamic & Static Frameworks/Libraries | Professional iOS Engineering Series
• A deep dive on macOS universal binaries
• Library Evolution in Swift
• xcframeworks repo

About xcconfig and build environment:

• Working With Custom “xcconfigs” and Cocoapods
• How to Set Up Xcode Build Configurations
• Custom xcconfigs
• Build Setting Reference
• TN3117: Resolving architecture build errors on Apple silicon
• Cocoapods: how to deal with device-only dependencies
• The Unofficial Guide to xcconfig files

About nanopb library — Nanopb: Basic concepts

A little bit about Kotlin Multiplatform Mobile:

• Get started with Kotlin Multiplatform Mobile
• Kotlin/Native iOS
• Setup Kotlin Native Project with CocoaPods
• Build final native binaries

A little more about the debugger:

• LLDB: Beyond “po”, WWDC’19
• Debug Swift debugging with LLDB, WWDC’22

Finally, a few radars and Apple forums, related to the topic:

• Module stability in Xcode 14?
• M1 Xcode 12.5 can’t paste clipboard
• Error when debugging: Cannot create Swift scratch context
• Archiving a Swift static library does not include Swift module

Good luck with moving to xcframeworks and feel free to discuss the paper in the comments! ;)