Open in app

Sign in

Write

Sign in

Mastodon

Xcode Build System: Compiling

Yeskendir Salgara
Kerege
Published in
12 min readJan 8, 2024

--

Understand how Xcode Build System decides what to compile and when. This article breaks down complicated terms and processes into simple explanations, perfect for anyone curious about what happens behind the scenes.

The overall picture of Compiling.

This article is a part of the Xcode Build System: Everything Everywhere All at Once main article.

After preparing for compilation, the Xcode Build System is now organizing a Swift module for the upcoming compilation phase.

Planning Swift module

Xcode Build System — Timeline: Planning Swift Module

SwiftDriver (LongNights normal arm64 com.apple.xcode.tools.swift.compiler)

cd LongNights
    builtin-SwiftDriver
        ~/Xcode.app/.../usr/bin/swiftc
    -module-name LongNights
    -Onone
    -whole-module-optimization
    -enforce-exclusivity\=checked
        @~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNights.SwiftFileList
    -DDEBUG 
    -sdk
        ~/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.0.sdk
    -target arm64-apple-ios17.0-simulator
    -enable-bare-slash-regex
    -g
    -module-cache-path
        ~/Xcode/DerivedData/ModuleCache.noindex 
    -Xfrontend
    -serialize-debugging-options
    -profile-coverage-mapping
    -profile-generate
    -enable-testing
    -index-store-path
        ~/Xcode/DerivedData/.../Index.noindex/DataStore
    -swift-version 5
    -I
        ~/Xcode/DerivedData/.../Debug-iphonesimulator
    -F 
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F 
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F 
        ~/Xcode/DerivedData/.../Debug-iphonesimulator
    -emit-localized-strings
    -emit-localized-strings-path
        ~/Xcode/.../LongNights.build/Objects-normal/arm64
    -c
    -num-threads 12
    -Xcc -ivfsstatcache
    -Xcc
        ~/Xcode/DerivedData/SDKStatCaches.noindex/iphonesimulator17.0-21A326-9f995cea1212cde75317d7d3cd2045e2.sdkstatcache
    -output-file-map
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNights-OutputFileMap.json
    -use-frontend-parseable-output
    -save-temps
    -no-color-diagnostics
    -serialize-diagnostics
    -emit-dependencies
    -emit-module
    -emit-module-path 
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNights.swiftmodule 
    -validate-clang-modules-once
    -clang-build-session-file
        ~/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation
    -Xcc -I
        ~/Xcode/.../LongNights.build/swift-overrides.hmap
    -emit-const-values
    -Xfrontend
    -const-gather-protocols-file
    -Xfrontend
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNights_const_extract_protocols.json
    -Xcc -iquote
    -Xcc
        ~/Xcode/.../LongNights.build/LongNights-generated-files.hmap
    -Xcc -I
        ~/Xcode/.../LongNights.build/LongNights-own-target-headers.hmap
    -Xcc -I
        ~/Xcode/.../LongNights.build/LongNights-all-non-framework-target-headers.hmap
    -Xcc -ivfsoverlay
    -Xcc
        ~/Xcode/.../LongNights.build/all-product-headers.yaml
    -Xcc -iquote
    -Xcc
        ~/Xcode/.../LongNights.build/LongNights-project-headers.hmap
    -Xcc -I
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/include
    -Xcc -I
        ~/Xcode/.../LongNights.build/DerivedSources-normal/arm64
    -Xcc -I
        ~/Xcode/.../LongNights.build/DerivedSources/arm64
    -Xcc -I
        ~/Xcode/.../LongNights.build/DerivedSources
    -Xcc -DDEBUG\=1
    -emit-objc-header
    -emit-objc-header-path
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNights-Swift.h
    -working-directory
        ~/LongNights 
    -no-emit-module-separately-wmo

The SwiftDriver is a component within the Swift toolchain responsible for coordinating the overall compilation process. It acts as a driver for the Swift compiler (swiftc), managing various tasks involved in compiling Swift source code.

The output of the builtin-SwiftDriver command itself, which invokes the Swift compiler (swiftc), typically includes information about the compilation process. This information is often extensive and may include details such as:

  • The actual command-line invocation of the Swift compiler, including the compiler executable path, options, and flags.
  • A list of Swift source files that are part of the current compilation task.
  • Information about the progress of the compilation, such as the stages being executed.
  • Paths where the compiled artifacts (object files, modules, etc.) are being generated.
  • Any diagnostic messages, warnings, or errors encountered during compilation.
  • Optionally, performance-related information or metrics may be included, especially if profiling or optimization features are enabled.
  • Depending on the specific build settings and options, additional details related to the build process may be present.

Here you can see that all the files(.hmap, .json, .SwiftFileList) that were generated before, now act as inputs.

The Swift compiler command is provided with various options and parameters.

  • -module-name LongNights: Specifies the module name for the Swift module.
  • -Onone: Disables optimizations.
  • -whole-module-optimization: Enables whole module optimization.
  • -enforce-exclusivity\=checked: Enables checking of exclusivity violations.
  • -enable-bare-slash-regex: Enables bare slash regex.
  • -g: Generates debug information.
  • ~/LongNights.build/Objects-normal/arm64/LongNights.SwiftFileList: Specifies the Swift file list.
  • -target arm64-apple-ios17.0-simulator: Sets the target architecture.
  • -Xfrontend -serialize-debugging-options: Instructs the frontend to serialize debugging options.
  • -profile-coverage-mapping -profile-generate: Enables code coverage profiling and generation.
  • -swift-version 5: Specifies the Swift language version.
  • -num-threads 12: Specifies the number of threads to use for compilation.
  • -output-file-map ...: Specifies the output file map path.
  • -no-emit-module-separately-wmo: Disables emitting modules separately.
  • -disable-cmo flag is used in Swift compilation to disable the "cross-module optimization" (CMO) feature. Cross-module optimization is a Swift compiler optimization that allows the compiler to optimize across the boundaries of Swift modules. When enabled, the compiler can perform additional optimizations that involve multiple modules, potentially resulting in better performance.

Let’s explore all the options provided in this command in more detail.

-Xfrontend flag is used to pass additional options directly to the Swift frontend, which is the part of the Swift compiler responsible for parsing, type-checking, and generating intermediate representations of the code. The -Xfrontend option allows you to customize the behavior of the frontend with specific options that are not part of the standard Swift command-line interface.

-Xcc:

  • -X flags in Swift are used to pass additional arguments to the compiler.
  • -Xcc specifically indicates that the following argument is intended for the C compiler.

In Swift compilation, the -emit flags are used to control the generation of specific output files or data during the compilation process. Here are a few commonly used -emit flags:

-emit-objc-header:

  • This flag instructs the Swift compiler (swiftc) to generate an Objective-C bridging header. The bridging header is used when you have Swift code in your project that needs to be interoperable with Objective-C code. The generated header file (-emit-objc-header-path) contains declarations for Swift entities that you want to expose to Objective-C.

-emit-module:

  • This flag tells the compiler to emit a binary module file (with a .swiftmodule extension). Module files contain information about the Swift module, which can be used for separate compilation and optimization. We’ll talk about it later.

-emit-module-path:

  • Specifies the path where the emitted module file should be saved.

-emit-const-values:

  • This flag instructs the compiler to emit constant values (for example, enum cases) into a separate file.

These flags give you control over what kind of output the Swift compiler produces during the compilation process. Depending on your needs, you might use these flags to generate specific files or data for further processing or to support interoperability with other languages or tools.

The overall picture of Planning Swift Module (.swiftmodule) with Inputs and Outputs.

As you may already notice there is an important option: -whole-module-optimization.

If we turn it off we see that there appears one task calling “Emmiting module”.

Xcode Build System — Timeline: Emitting module (if whole-module-optimization turned off)

It means that in order to produce .swiftmodule file it requires more information of all the swift files compilations.

Whole Module Optimization (WMO) is an advanced compiler optimization technique used by the Swift compiler to generate highly optimized code for an entire Swift module. It works by analyzing and optimizing the entire module as a single unit, rather than compiling individual source files separately. Here’s how it works:

When you enable WMO, the Swift compiler first performs a deep analysis of the entire module. It considers all the code within the module, including functions, types, and dependencies between them. This analysis involves understanding the interactions between different parts of the code.

With knowledge of the entire module, the compiler can perform more aggressive and global optimizations. It can identify opportunities for optimization that would be missed when compiling individual source files independently.

One of the key optimizations enabled by WMO is inlining. Inlining involves replacing a function call with the actual code of the function wherever it’s used. This eliminates the overhead of function calls and can lead to significant performance improvements. WMO can perform inlining across module boundaries, which is not possible when compiling files separately.

WMO allows the compiler to analyze and optimize code across different source files within the module. This means that it can make optimizations that involve interactions between code defined in different files. For example, it can optimize code that passes complex data structures or objects between functions in different files.

It can optimize data structures, eliminate redundant computations, and perform other high-level transformations.

WMO can perform more effective dead code elimination. It can detect code that is never used within the entire module and remove it from the final binary, reducing the size of the executable.

After the global analysis and optimizations, the Swift compiler generates highly optimized machine code for the entire module. This optimized code is what gets included in your final application or framework.

The main benefit of WMO is improved runtime performance. The highly optimized code generated by WMO can lead to faster execution of your Swift code, making your application or framework more responsive and efficient.

Compiling swift files

Xcode Build System — Timeline: Compiling .swift files
cd LongNights
    builtin-swiftTaskExecution
        ~/Xcode.app/.../usr/bin/swift-frontend
    -frontend -c
        ~/LongNights/Filters/FilterSolutions.swift
        ~/LongNights/ContentView.swift
        ~/LongNights/LongNightsApp.swift
        ~/Xcode/.../LongNights.build/DerivedSources/GeneratedAssetSymbols.swift
    -supplementary-output-file-map
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/supplementaryOutputs-49
    -emit-localized-strings
    -emit-localized-strings-path
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64
    -target arm64-apple-ios17.0-simulator
    -Xllvm -aarch64-use-tbi
    -enable-objc-interop
    -sdk
        ~/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.0.sdk
    -I
        ~/Xcode/DerivedData/.../Debug-iphonesimulator
    -F
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F
        ~/Xcode/DerivedData/.../Debug-iphonesimulator/PackageFrameworks
    -F
        ~/Xcode/DerivedData/.../Debug-iphonesimulator
    -no-color-diagnostics
    -enable-testing
    -g
    -module-cache-path
        ~/Xcode/DerivedData/ModuleCache.noindex
    -profile-generate
    -profile-coverage-mapping
    -swift-version 5
    -enforce-exclusivity\=checked
    -Onone
    -D DEBUG
    -serialize-debugging-options
    -const-gather-protocols-file
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/LongNights_const_extract_protocols.json
    -enable-bare-slash-regex
    -empty-abi-descriptor
    -validate-clang-modules-once
    -clang-build-session-file
        ~/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation
    -Xcc
    -working-directory
    -resource-dir
        ~/Xcode.app/.../usr/lib/swift
    -enable-anonymous-context-mangled-names
    -Xcc -ivfsstatcache
    -Xcc
        ~/Xcode/DerivedData/SDKStatCaches.noindex/iphonesimulator17.0-21A326-9f995cea1212cde75317d7d3cd2045e2.sdkstatcache
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/swift-overrides.hmap
    -Xcc -iquote
    -Xcc
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/LongNights-generated-files.hmap
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/LongNights-own-target-headers.hmap
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/LongNights-all-non-framework-target-headers.hmap
    -Xcc -ivfsoverlay
    -Xcc
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/all-product-headers.yaml
    -Xcc -iquote
    -Xcc
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/LongNights-project-headers.hmap
    -Xcc -I
        ~/Xcode/DerivedData/LongNights-exbpdzmtpimctvbptilfftsijotl/Build/Products/Debug-iphonesimulator/include
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/DerivedSources-normal/arm64
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/DerivedSources/arm64
    -Xcc -I
        ~/Xcode/.../LongNights.build/Debug-iphonesimulator/LongNights.build/DerivedSources
    -Xcc
    -DDEBUG\=1
    -module-name LongNights
    -frontend-parseable-output
    -disable-clang-spi
    -target-sdk-version 17.0
    -target-sdk-name iphonesimulator17.0
    -external-plugin-path
        ~/Xcode.app/.../iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.0.sdk/usr/lib/swift/host/plugins\#~/Xcode.app/.../usr/bin/swift-plugin-server
    -external-plugin-path
        ~/Xcode.app/.../iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator17.0.sdk/usr/local/lib/swift/host/plugins\#~/Xcode.app/.../usr/bin/swift-plugin-server
    -external-plugin-path
        ~/Xcode.app/.../iPhoneOS.platform/Developer/usr/lib/swift/host/plugins\#~/Xcode.app/.../usr/bin/swift-plugin-server
    -external-plugin-path
        ~/Xcode.app/.../iPhoneOS.platform/Developer/usr/local/lib/swift/host/plugins\#~/Xcode.app/.../usr/bin/swift-plugin-server
    -plugin-path
        ~/Xcode.app/.../usr/lib/swift/host/plugins
    -plugin-path
        ~/Xcode.app/.../usr/local/lib/swift/host/plugins
    -num-threads 12
    -o
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/FilterSolutions.o
    -o
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/ContentView.o
    -o
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/LongNightsApp.o
    -o
        ~/Xcode/.../LongNights.build/Objects-normal/arm64/GeneratedAssetSymbols.o
    -index-unit-output-path
        ~/LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/FilterSolutions.o
    -index-unit-output-path
        ~/LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/ContentView.o
    -index-unit-output-path
        ~/LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/LongNightsApp.o
    -index-unit-output-path
        ~/LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/GeneratedAssetSymbols.o
    -index-store-path
        ~/Xcode/DerivedData/LongNights-exbpdzmtpimctvbptilfftsijotl/Index.noindex/DataStore
    -index-system-modules

Here we see that all swift files (including asset symbols):

  • FilterSolutions.swift, ContentView.swift, LongNightsApp.swift
  • GeneratedAssetSymbols.swift

The output of this command include compiled object files (.o files). These object files are intermediate files generated during the compilation process and are used later in the linking phase to produce the final executable.

Additionally, the compilation process may produce diagnostic messages, warnings, and errors, which would be printed to the console during the execution of this command.

Let’s consider options of this command:

  • -Xllvm flag in Swift compiler commands is used to pass LLVM options directly to the LLVM backend during the compilation process.

When you use the -Xllvm flag, you can provide specific LLVM options to fine-tune the code generation or perform other low-level optimizations.

  • -aarch64-use-tbi is an LLVM option that enables the use of the Top Byte Ignore (TBI) feature on the AArch64 (ARM64) architecture. TBI is a feature in ARM architectures that ignores the top byte of addresses, allowing more efficient use of the address space.
  • -disable-clang-spi flag is used with the Swift compiler (swiftc) and is related to Swift's interaction with Clang, the C and C++ compiler front-end often used in conjunction with Swift when developing mixed-language projects.

SPI stands for “Swift Private Interfaces.” SPIs are a way for Swift to interact with certain implementation details of a library or framework, even though those details are not part of the library’s public API.

In other words, it disables the Swift compiler’s ability to access these private interfaces. This can be useful in scenarios where you want to ensure that your Swift code doesn’t rely on or access any internal implementation details of C or C++ libraries.

Using this flag can help maintain code stability and compatibility because it prevents Swift code from relying on features that may change or be removed in future library updates, promoting better code encapsulation and reducing the risk of breaking changes when working with C or C++ libraries from Swift. However, it can also limit your ability to interact with certain library features that are not part of the public API.

  • -enable-anonymous-context-mangled-names it is related to how Swift generates mangled names for entities within the code. Mangled names are used to uniquely identify entities such as functions, methods, types, and variables in the compiled binary, especially when dealing with overloads, generic types, and nested types.

When you enable this flag, you are instructing the Swift compiler to include information about the lexical context (e.g., the containing type or scope) in the mangled names for certain entities.

Including the lexical context in mangled names can make it easier to understand error messages and debugging output. When dealing with generic types or nested types, it can be more informative to have mangled names that reflect the context in which these entities are defined.

It helps avoid name clashes, especially when you have multiple entities with the same name within different generic contexts or nested scopes. By including the context in the mangled name, Swift ensures that these entities have distinct names in the binary.

The overall picture of compiling with a whole-module-optimization option

As I mentioned the compiling process depends on the whole-module-optimization option (Build Settings). In the case when this option is turned off the compiling task of swift files runs in parallel.

Xcode Build System — Timeline: Compiling (whole-module-optimization turned off)

Unlock downstream dependents

Xcode Build System — Timeline: Unlock downstream dependents of Target

This task using builtin-Swift-Compilation-Requirements .

It’s checking if it’s safe to proceed with compiling downstream dependents.

Swift Merge Generated Headers

Xcode Build System — Timeline: Merge Objective-C generated interface headers
cd ~/LongNights
    builtin-swiftHeaderTool 
  -arch arm64 
    ~/Xcode/DerivedData/.../Intermediates.noindex/.../LongNights.build/Objects-normal/arm64/LongNights-Swift.h 
  -o 
    ~/Xcode/DerivedData/.../Intermediates.noindex/.../LongNights.build/DerivedSources/LongNights-Swift.h

This command is essentially telling Xcode’s built-in tool to generate or update the Swift header file for the project targeting the arm64 architecture. This generated header file will be used to bridge between Swift and Objective-C code within the project, allowing each language to access the other's APIs as needed.

Copy .abi.json file

cd /Users/U780052/Desktop/LongNights
    builtin-copy 
  -exclude .DS_Store 
  -exclude CVS 
  -exclude .svn 
  -exclude .git 
  -exclude .hg 
  -resolve-src-symlinks 
  -rename 
    ~/Xcode/DerivedData/.../Build/Intermediates.noindex/.../LongNights.build/Objects-normal/arm64/LongNights.abi.json 
    ~/Xcode/DerivedData/.../Build/Products/Debug-iphonesimulator/LongNights.swiftmodule/arm64-apple-ios-simulator.abi.json

This command is copying a JSON file containing ABI (Application Binary Interface) information related to the project from a directory of built products to an intermediate directory. The ABI file is essential for the Swift compiler and runtime to ensure compatibility and correct behavior of the binary code.

ABI details are crucial in ensuring that the compiled code correctly interacts with other compiled code (e.g., libraries, runtime, etc.), especially when there are updates or changes to the Swift language or the project itself.

  • -resolve-src-symlinks: This option indicates that if the source file is a symbolic link, the command should resolve it to the real file before copying.

Copy .swiftmodule file

cd ~/LongNights
    builtin-copy 
      -exclude .DS_Store 
      -exclude CVS 
      -exclude .svn 
      -exclude .git 
      -exclude .hg 
      -resolve-src-symlinks 
      -rename 
          ~/Xcode/DerivedData/.../Build/Intermediates.noindex/LongNights.build/Debug-iphonesimulator/LongNights.build/Objects-normal/arm64/LongNights.swiftmodule 
          ~/Xcode/DerivedData/.../Build/Products/Debug-iphonesimulator/LongNights.swiftmodule/arm64-apple-ios-simulator.swiftmodule

The .swiftmodule file contains the necessary metadata about the module, allowing other modules or the main app to use the compiled module without needing access to its source code.

The .swiftmodule file acts as an interface for the module. It includes information about the public types, functions, variables, and other members defined in the module. When you import a module in Swift, the compiler reads the .swiftmodule file to understand what is available from that module and ensure that your code interacts correctly with it.

During the compilation of a program that uses the module, the Swift compiler uses the .swiftmodule file for type checking and to generate code. It ensures that the types and function signatures match what is expected and generates the appropriate calls to the module's code.

Swift’s powerful generics system can be optimized using information in .swiftmodule files. The compiler can use the metadata about generic types and functions to generate specialized versions of the generics for the actual types they are used with in your program, which can greatly improve performance.

In larger projects, or projects that depend on several modules, .swiftmodule files are essential for incremental builds. The compiler can skip recompiling a module if the .swiftmodule file indicates that there have been no changes. This significantly speeds up the build process, especially during development when changes are being made frequently.

The Swift compiler can use the information in .swiftmodule files to perform optimizations across module boundaries. For example, if a public function in a module isn't actually used by any external code, the compiler might be able to omit that function from the final binary, even if the function is technically "public" within the module's scope.

This article is a part of the Xcode Build System: Everything Everywhere All at Once main article, where you can read more about Linking.

Buy a Coffee for salgara: https://ko-fi.com/salgara

Compilation
Swift
Xcode

--

--

Yeskendir Salgara
Kerege

Written by Yeskendir Salgara

56 Followers

Software Engineer, Writing about Swift and ML

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams