Our CI builds several iOS frameworks which are then deployed into Artifactory, ready for consumption by our clients.
Recently, we had a framework crash when used by our client, naturally, we wanted to attach the offending app to the debugger and step into our code,
but alas, which we couldn’t trivially do because Xcode doesn’t have a simple way of telling where the symbols are and where the sources directory is at.
Naturally I went ahead to Google, then when I couldn’t find a proper solution I turned to StackOverflow in which I put a bounty on the question.
Some time later, I received a reply by a guy that goes by the name Jim Ingham, and after a quick search out of curiosity turns out he’s one of LLDB’s principal engineers from Apple
(awesome — first order source of knowledge!).
A Little About Symbols on iOS/macOS
At first I was under the misconception that it’s enough that symbolic and source map data is embedded into the debug binaries of our frameworks, but thats not enough.
In fact, that data is read from either of these two places:
- .o intermediates object files on the machine the frame was compiled on (the CI machine in our case).
These aren’t portable (unless you hack around it — which you shoudln’t)
- dSYM packages which are generated if DEBUG_INFORMATION_FORMAT is set to dwarf-with-dsym.
This is portable and usable, however, it had absolute paths to the sources embedded in it.
Attaching a dSYM manually
After getting the answer in StackOverflow, I went ahead and tried doing a manual process of attaching sources, its as simple as follows:
- I had to have the dSYM on my machine.
- I started Xcode
- Set a breakpoint
- Launch the app
- When the breakpoint got hit, I’ve typed the following commands in lldb console:
image lookup -vn SomeGlobalSymbolz
This printed out the absolute path of the source code file in which that symbol resides e.g:
Now i had to remap the sources path so I used
settings set target.source-map /Users/JENKINS/MyProject/src/SomeGlobalSymbol_Source.cpp /Users/Max/MyProject/src/SomeGlobalSymbol_Source.cpp
- Note — you can map multiple source-maps thus:
settings set target.source-map tree1 mapped-tree1 tree2 mapped-tree2 …
6. I was now able to step in the source of symbols in that binary framework and debug properly.
The process in the previous section it tedious, not a fun use of my nor your time, so I automated it.
You can grab the Python 2.7.x script from https://gist.github.com/GoatHunter/cf85b37eac2820ecb9a72792bc9011ef
Here’s a usage example:
python dsym_symbolizer.py — dsym_url https://www.example.com/MyStuff.framework.dSYM.zip \
— source_path /Users/MeMySelfAndI/MyStuffSources \
A Ruby version of the script can be found here.
- Thanks to my fellow coworker Ben Asher for making the Ruby port.
- Note that the Ruby port requires the dSym
Now, the automation actually doesn’t do whats written in the previous section, what it does do is actually whats described in http://lldb.llvm.org/symbols.html
with a guesstimation thrown in –
The script looks into the binary, and tries to match an embedded path to a local path using the following algorithm:
- It downloads the dSYM from a remote URL.
- dSYMs are put into ~/dsyms
- Executes the nm utility on the binary and guestimates the embedded sources path using the following logic:
3.1. Let’s assume that source_path == /my/path
3.2. potential_original_path == /remote/place/foo/bar/baz
3.3. Then we attempt to see if /my/path/baz exists, if not then /my/path/bar/baz, and then /my/path/foo/bar/baz and if it does we return /remote/place/
4. It then generates a mapping plist for each architecture with mapping data
and puts them into the Contents/Resources within the dSYM package.
Here’s an example plist:
<?xml version=”1.0" encoding=”UTF-8"?>
<!DOCTYPE plist PUBLIC “-//Apple//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd">
5. Now when you run Xcode, lldb will automatically pick out the dSYM and let you debug the binary on your machine!
– Note — you must re-run the script only if the binary gets changed!
Update July 2020
There’s also a Ruby port of the script here. Thanks goes to my co-worker Ben Asher.
Note that the Ruby port requires the dSYM to be a local path.