How to Debug Swift Compiler: Part 1 — Environment

Image by Sammy-Sander from https://pixabay.com/photos/thinking-mentor-mindset-detective-4125016/

Learn how to leverage the open-source nature of Swift compiler and contribute for everyone’s benefit!

When I was enabling linux support in Sourcery, I wondered how exactly I could debug some of the bugs in the Swift compiler I faced.

Sure enough, it was not an easy task. Let’s dive-in!

Environment Setup

To debug Swift compiler, first we need to understand how it works.

Swift compiler consists of two parts: driver and frontend , where driver is inspired by the design of Clang, and frontend might not even exist at all. Quoting the official documentation mentioning swift-frontend :

The frontend is an implementation detail. No aspects of its command-line interface are considered stable. Even its existence isn’t guaranteed

Swift Driver

To simplify, swift driver “drives” the process of compilation. It is not hardware-related, but just a “build pipeline controller”. It controls parsing, pipelining, binding (i.e. ld main.o -o main) and execution — spawning all of the sub-processes of the build.

More info about swift driver internals is available here.

Swift Front-end

Constructing AST, type checking, matching types between each other, generating assembly, LLVM IR, scanning dependencies, recognizing imports — these are just some tasks that Swift front end does. As an example, Xcode invokes swift frontend when compiling Swift source files:

Example of Xcode compilation log which simply invokes `swift-frontend` with a list of files per build process’ segment

What to Debug — swift driver or swift-frontend?

In most cases, a software developer would debug and fix issues in swift-frontend . At least that was my intention, to dive into the infamous error:

The compiler is unable to type-check this expression in reasonable time; try breaking up the expression into distinct sub-expressions

For that, I needed to run the swift-frontend in DEBUG mode, meaning with an attached LLDB.

I started by reading the github.com/apple/swift/docs/HowToGuides/GettingStarted.md. Little that I knew, it would take only 1 hour to build & run the Swift compiler itself, directly from Xcode! But could it have been done faster than this?

Swift compiler in Xcode

It is important to understand that Xcode comes with a built-in Swift compiler, located under Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin path, which cannot be replaced with a debug-able version checked out from GitHub:

Default location of a built-in Swift compiler inside of Xcode.app

So, what would be the approach? Having experience working on Xcode source code editor extensions using XcodeKit, I had an idea that I would somehow invoke Swift compiler with an argument list from a kind-of Workspace within the Xcode. And, in fact, that was the way to go!

Following the mentioned above GettingStarted.md , I have done the following:

git clone git@github.com:apple/swift.git swift
cd swift
utils/update-checkout --clone-with-ssh

It takes a while, for example, on M1 Max with 32 GB RAM, this setup process took up to 30 minutes in total. Script sets up the entire working directory for the Swift compiler:

Short snippet of Swift compiler’s dependencies checked out by the mentioned script

The next step is to execute brew install cmake ninja sccache. In order to compile Swift compiler, the following command needs to be executed:

utils/build-script --skip-build-benchmarks \
  --skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "$(uname -m)" \
  --sccache --release-debuginfo --swift-disable-dead-stripping

Ninja utility would generate xcodeproj located at swift-project/build/Ninja-RelWithDebInfoAssert, which you would then add to an xcworkspace:

Generated Swift project added to a new, manually created, workspace

Here comes the interesting part: how to actually run Swift compiler with a debugger attached, and debug it when it is going to be compiling code in one of your projects? Should you pass a single file? Should you invoke the -typecheck option of the swift-frontend?

Input for swift-frontend

Let’s see what are the commands swift-frontend accepts by running swift-frontend --help :

Supported flags of swift-frontend command

While there are so many interesting commands/options, which one might be the most important for you? The one which compiles the code and generates errors in Xcode? The one which generates AST? The one which runs the TypeChecker?

While it might seem straightforward, when it comes to being a beginner, it is very unclear which command does what. For example, -typecheck option would only check types for methods and variables within the given input, but method bodies / closures would not be parsed / type-checked. Then, how to invoke swift-frontend the same way as Xcode does?

Well, the answer lies within the question itself: we want to invoke swift-frontend the same way as Xcode does. And I mean exactly the same way. In the image before I have already shown that Xcode invokes swift-frontend which is embedded into the macOS toolchain:

Image from Xcode log navigator which shows invocation of swift-frontend

If I were to pass a single SyntaxVisitor.swift file to swift-frontend with an option -typecheck , parsing would fail, because not all types used in SyntaxVisitor.swift are defined inside that same file. We need to supply all files which contain types used in SyntaxVisitor.swift .

What I did not realise from the beginning was the fact that in order to compile the given input files fully, including method bodies, I needed to literally copy and paste Xcode log navigator’s invocations of swift-frontend . In my case I was debugging a file called SyntaxVisitor.swift while investigating a bug, which I have reported before when I was optimizing the TypeChecker’s performance for swift-syntax package. Luckily, Xcode already generates the list of files to be passed to swift-frontend for us:

-c /Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/generated/SyntaxVisitor.swift
/Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/AbsolutePosition.swift 
/Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/AbsoluteRawSyntax.swift 
/Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/AbsoluteSyntaxInfo.swift 
/Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/Assert.swift 
...
...
/Projects/SwiftTypeChecker/swift-syntax/Sources/SwiftSyntax/generated/syntaxNodes/SyntaxNodesTUVWXYZ.swift

Passing file paths this way will make swift-frontend parse and compile the whole file with method bodies, closure contents etc. like it would be done in Xcode. And so, I was able to reproduce the same referenced issue, while having a debugger attached!

Screenshot of Xcode with a triggered breakpoint inside of `swift-frontend` source code

Now, the only thing left is to actually fix bugs I have discovered, and so, see you in the next part of the series!

Want to Connect?

Follow me on X (Twitter): @r_alikhamov or LinkedIn 🤝

I like the Swift programming language. It is much more safer than Objective-C, even though it is and will, for a very long time, be as vulnerable, due to interoperability with Cocoa classes. Nonetheless, Swift compiler is mostly written in C++. In order to understand the behaviour of the Swift compiler, as a developer, I need to have the “baggage of knowledge” in many different domains. It is very beneficial to know, or, at least, be able to read other programming languages, no matter which preference a developer has, be it a compilable or an interpreted language model.

References

1. GettingStarted.md — https://github.com/apple/swift/blob/main/docs/HowToGuides/GettingStarted.md
2. Debugging The Compiler.md — https://github.com/art-divin/swift/blob/main/docs/DebuggingTheCompiler.md#debugging-the-type-checker
3. Swift Driver Design & Internals — https://github.com/apple/swift/blob/main/docs/DriverInternals.md
4. The Swift Driver — https://github.com/apple/swift/blob/main/docs/Driver.md