Inspecting Xcode’s build system graph at ease
How does the Xcode build system work under the hood?
If you ever tried to understand why the Xcode build system complains about a compilation cycle or optimize a build where a particular step is invalidated between executions, there aren’t any good visualization tools that might help with troubleshooting other than build logs or Build Timelines added in Xcode 14. While these two techniques might still give some insights, 1) they are only available as a post-build artifact so aren’t useful to troubleshoot build system dependency cycles and 2) these approaches have limitations.
From what I observe, many engineers underestimate the complexity of build systems. That is especially the case for Xcode users where predefined templates bootstrap the project magically and developers don’t have to set up steps manually.
In this blog post, we will scratch the surface of the Xcode build system and audit what steps are involved when you press ⌘+R in a visual representation of a build graph using XCBuildAnalyzer.
Scenario I — build cycle
For sure you have seen an error saying that you have a cyclic dependency in your project, which might be just a simple oversight with incorrectly set target dependencies, but in more complex scenarios (mixed ObjC&Swift target, I am looking at you), the fix might not be simple. If that happens, Xcode provides a simplified cycle detail in the Report navigator view and prints the “raw dependency cycle trace” below.
The latter data is quite difficult to analyze but gives us some clues about the internal implementation of the build system. It consists of all nodes and commands that constitute a cycle in the build system graph.
Commands are actual steps that have a concreted action like calling an external application, while nodes are just virtual nodes, convenient for setting dependencies between commands. You can think of commands as real actions and nodes as breakpoints describing what kind of outputs/inputs a given command produces/depends on. A list of sample commands and nodes below might give you a clue of their differences:
Command (with tool involved):
Copy(file-copy)SwiftDriver Compilation(swift-driver-compilation)SwiftMergeGeneratedHeaders(swift-header-tool)WriteAuxiliaryFile(auxiliary-file)PhaseScriptExecution(shell)CreateBuildDirectory(create-build-directory)ClangStatCache(shell)VersionPlistTaskProducer(phony)CodeSign(code-sign-task)
Nodes:
entrybegin-scanningbegin-compilingcopy-headers-completion
In our example, a quick inspection of all nodes and commands provides an overview of things happening. Names refer to module map/header map creation, validation, assets compilation, and other terms that not always might be obvious. To have a better mapping of all relations between steps, having a graphical visualization of all these nodes might be helpful. Fortunately, for each build (including those that failed with a cycle error), Xcode generates a full dump of the build graph nodes in the DerivedData. Despite the representation being in a readable JSON format, navigating through a large object file might be tricky, which is why I created an experimental macOS app, XCBuildAnalyzer, that draws it on a 2D plane.
The build analyzer can draw a graph that might look familiar to you if you have ever troubleshooted memory cycles in Xcode or Instruments. The goal of this app is to draw the graph in a digestive format, easier to create a mental model of parts in a cycle. Download the app and drag and drop the .xcodeproj or Package.swift after building it in Xcode.
As you may guess, the graph itself is very big so drawing it on a screen wouldn’t be helpful — thus, the app presents a subgraph, similar to the one generated for our example.
If you follow the yellow cycle path, you find that Library1 depends on Library2 in links:
[Library1] end->[Lirary2] entry[Library2] modules-ready->[Library1] begin-compilation
That cycle is something we have to resolve — there aren’t any ready-to-apply fixes — problems are project-specific and the analysis should be carried over case-by-case.
For details, the XCBuildAnalyzer app allows expanding nodes in the graph to disclose steps relevant to troubleshooting.
Tip: the build graph view can very easily get out of control as you expand nodes. If that happens, just start the process from the beginning, just like you would do in the Xcode memory inspector.
Scenario II — missing dependencies
Do you recall a problem with random and transient Swift errors happening after cleaning a project or building on a different machine with the message No such model {HereModule}? That might be caused by incorrectly defined target dependencies and based on a nondeterministic order of compilation, the required {HereModule} might, or might not be ready yes.
If your project contains only a few targets, probably you can quickly find a missing link, but for bigger projects with dozens or hundreds of targets, that might be tricky.
The first sign of a problem can surface right after opening the graph and realizing that nodes from the problematic target {HereModule} are missing. That implies that you haven’t set any dependency to that module and it works on your machine because you built that target earlier or using a different scheme. But if both targets are on a list, let’s find the first common node that depends on two targets. First, we have to find which compilation target reported an error. In Xcode’s report navigator pick Recent and Errors only selectors to filter all other steps:
Top2 module (the target name is likely also Top2)To inspect a problem, we want to find a relation between build graph nodes “finishing the Top2” and “beginning of the Down3“. Let’s open XCBuildAnalyzer and with the ⌘key, select [Top2] modules-ready and [Down3] begin-compiling. Let’s analyze the result below:
Let’s start by reading the graph from right to left. The build system always starts the analysis from theend node (0.) and traverses through the graph to find the steps required for execution.
Firstly, we can realize that there is no direct or indirect relation between Down3 (1.) and Top2 targets (2.). So why isTop2 even included in the build and why the build system is not complaining all the time? As seen, Top3 (3.) is the target that has an explicit dependency on a Top2, and based on the compilation order, Top2 and Top3 might be compiled prior to Down3.
Note that even the graph renders
Down3 begin-compillingto the right of[Top2] modules-ready, that doesn’t mean it will be executed later. Remember that the graph represents relations between nodes; it does not represent a timeline.
Once we add a missing dependency in Xcode and compile the project again, the subgraph reveals the expected dependencies between Down3 and Top2:
Scenario III — curiosity
The last reason why you may want to review the build graph is pure curiosity. For instance, Xcode 15 introduced a nice feature to generate Swift/ObjC type with all of your Assets symbols (for more details, read this blogpost). You might ask yourself: (1) how is that implemented? and (2) can I safely modify .xcassets content from prebuild shell build scripts so the generated asset symbols file will always reflect the change?
Finding a starting point for the assets symbols generation shouldn’t be difficult because node names are often self-explanatory (like CompileAssetCatalog, GenerateAssetSymbols , TestTargetTaskProducer, etc.). In our case, that will be GenerateAssetSymbols and on the right pane of the XCBuildAnalyzer, you can find that it is an external shell command (actool) invocation. Bonus: you can inspect the list of args and ENVs passed to that external command.
As expected, this step will output two files (ObjC and Swift) and take Assets.xcassets directory as input.
Question 2: Can I safely modify .xcassets content from pre-build shell build scripts so the generated asset symbols file will always reflect the change?
For the second question, before jumping into heavy 2D graph visualization, let’s add a shell script and analyze raw Xcode’s build output logs to see which one was executed first.
In the logs, seems that our script was indeed scheduled first, but is that a coincidence? Maybe we were just lucky? Let’s open the project in XCBuildAnalyzer and compare the relationship between [GenerateAssetSymbols] xxx.swift and <execute-shell-script> . From the left pane select two nodes (you can use ⌘ or ⇧ keys) and you will get something like:
Success! So there is a strong relation between those two steps and we can be confident that GeneratedAssetSymbol.swift will be generated after the script is done. Saying that we can safely modify the .xcassets directory, and the actoolwill respect it.
Warning! Having the right asset symbol in Swift/ObjC is just a partial success. To ensure that the modified assets will be included in the application bundle, we need to ensure that the assets compilation step (
CompileAssetCatalog) is also dependent on the run script. With just a few clicks you can validate that this is also a case —Assets.car(the compiled assets representation) will also include updated assets.
Epilogue
XCBuildAnalyzer is one of those tools that sit in your toolbelt and you hope to never need it. However, it might be helpful if you hit a tricky build system problem that you cannot resolve in a conventional way: by reading logs or double-checking Xcode target dependencies.
So let’s hope for the best, but prepare for the worst — try out XCBuildAnalyzer and say hello to the Xcode build system internals!
