Fun with rpath, otool, and install_name_tool

Some recent work has forced me to become significantly more friendly with native library resolution on macOS. I’ve been meaning to write down what’s I’ve learned, and a question on the Xamarin.Mac forum encouraged me to make time.

Before I begin two caveats:

• In theory, Xamarin.Mac’s packaging tooling should handle this for you. If you hit cases that obviously should work and require manual tweaking, please file an issue with an example project.
• I am far from an expert in this area and this is a high level overview. You can read more details here and this is a solid reference page. I was unable to find a nice general overview anywhere, which is why I wrote this up. Feel free to leave corrections in the comments.

When macOS loads your executable, it will also load any dynamic libraries (and frameworks) directly depended upon. Most of the time this happens without much notice (and way before your code executes) but when it fails you end up with a frightening message like this:

dyld: Library not loaded: MyLibrary
  Referenced from: /path/to/my/application.app/Contents/MacOS/application
  Reason: image not found

Let’s dig into what’s going on. Unlike Windows, which will look in a number of locations like next to your exe, macOS will not (by default). Embedded in your executable are the loader instructions that it follows exactly. otool -lwill dump them in a mostly human readable format.

The output will list a large number of load commands, today we’re primarily interested in LC_LOAD_DYLIB and LC_RPATH.

• LC_LOAD_DYLIB says “Please load a native library at this path”. It can be an absolute path (/usr/lib/libSystem.B.dylib), relative (../libFoo.dylib) or special (@rpath/Foo.framework/Foo).
• LC_RPATH adds items to the rpath, which we’ll describe shortly.

Absolute paths are obvious, and are only really common for system libraries. Relative paths are often wrong, as they are based on your current directory (which means if you run it from another location it may not work).

Often you’ll want to use a location relative to the executable, which is what some of the special “@” paths are for. There are a number of different options:

@executable_path, @loader_path, @rpath

with this is a reasonable reference.

rpath is the most important for our purposes because it says “look for the library in every folder specified with a LC_RPATH command. This allows, for example, the main executable to setup an rpath and dependent libraries to consume it, without knowing where it necessarily is.

Now that you have a rough understanding of how the library loading words, how can you influence it? There are two common approaches:

• Pass linker commands to clang (often of the form -Xlinker option -Xlinker value)
• Use install_name_tool to modify things post build.

install_name_tool has a number of options, but the two I commonly use are:

install_name_tool -add_rpath @executable_path/. a.out

which as it sounds adds whatever path (in this case next to the binary) to my a.out

install_name_tool -change libFoo @rpath/libFoo a.out

and this which changes the loader instruction for libFoo from it’s old value (libFoo) to a new one (@rpath/libFoo)

There are also a number of environmental variables that you can set to have the native loader output information. Sometimes dumping the list of what libraries were resolved from what location:

DYLD_PRINT_LIBRARIES=1 ./Foo.app/Contents/MacOS/Foo

can be enlightening.