Using LLVM for seamless interop with Clang languages

Image for post
Image for post
LLVM is the magic glue that will let us easily implement interop.

Recently I’ve been working on a project, VSL which is a bare-metal, high-level, compiled language. The problem is with new languages, particularity obscure ones made in someone’s spare-time is it lacks a community or ecosystem. On the contrary you have languages such as C, C++, Swift, and Objective-C which have passionate communities and libraries.

Interoperability is the approach VSL has choosen. They are a couple languages that have native interoperability with other languages, first ones coming to mind are D, and Swift. D takes a more ABI-compatiblity approach while Swift uses the same runtime as the language it interfaces with. How can two distinctly different languages communicate? We can thank LLVM for this.

Just to be familiar with VSL syntax before I start. Something like this (VSL):

public class UIViewController {
public func viewDidLoad() external(symbol_name);

declares a function UIViewController with method viewDidLoad . viewDidLoad ‘s body however is ‘external’ so the behavior of it will be some other function (named symbol_name ) that we’ll link to.

They are a couple things which we’ll note that should work:

  • Dynamic Dispatch
  • alloc/dealloc overrides
  • super interaction
  • Subclassing & Implementation
  • Type bridging e.g. NSString to a VSL String (I won’t discuss this but it’s relatively trivial to add)

All of these need to work in both languages. Now the basic idea behind using LLVM is generating the appropriate calling code by bridging between both ABI/calling convention differences. Let’s try declaring a function that allows a C-conformant language to call the Objective-C UIView#addSubview method:

Compiling this (with the vsl-objc-bindgen tool) we get:

(with optimizations to remove dead code)

This way we let clang do all the heavy lifting and get us our calling code. (the #0 refers to the alwaysinline attribute and a few others). With our calling code, we see that Clang has generated the messaging to the Objective-C functions. Now let’s write a VSL class that implements this:

public class UIView: OpaquePointer {
public func addSubview(view: UIView) external(UIView_addSubview)
func main() {
let view: UIView = UIView()
let subview: UIView = UIView()

VSL here generates an external method for the subview as:

Now once we link all of these together we get code that calls the Objective-C method!

We’ll run the following build commands:

$ vsl-objc-bindgen compile ios iphonesimulator UIView.addSubview.m -F UIKit
$ vsl build UIView.addSubview.vsl -S -o UIView.addSubview.vsl.ll -T x86_64-apple-ios11.3.0
$ llvm-link artifacts/vsl-x86_64-apple-ios11.3.bc UIView.addSubview.vsl.ll -o=- | opt -o=- -O2 | llvm-dis

Looking at this you’ll notice we’re doing @malloc(i64 0) which will give us a null pointer so this code will immediately crash. That is undesirable so we will now need to interface our initializer. We can do that by doing a similar thing, this time calling [[UIView alloc] initWithFrame: cgRect] . For reference, CGRect has size 32 so we’ll use a UInt32 in VSL so we don’t need to bridge the CGRect struct yet. Let’s write the code for this:

Note: CGRect is a ‘struct’ but it actually refers to struct CGRect*

And again compiling this we get:

We’ll create a external call in VSL:

public typealias CGFloat = Double
public typealias CGRect = Pointer<UInt32>
public func CGRectMake(x: CGFloat, y: CGFloat, width: CGFloat, height: CGFloat) -> CGRect external(CGRectMake)public class UIView {
public init(with frame: CGRect) external(UIView_initWithFrame)

// ...

Now we’ll call the initializer along with our previous setSubview :

func main() {
let cgRect = CGRectMake(x: 0, y: 0, width: 0, height: 0)

let view = UIView(frame: cgRect)
let subview = UIView(frame: cgRect)

(we can create bindings for the first two lines but right now we’ll just create a CGRect with all fields as zero). Compiling this again now we get:

Joining this with the previous LLVM results we can finally get a full UIView example:

So as you can see in the above we have been able to interop between VSL and Objective-C without any runtime overhead. In the above example we don’t do memory management but you can link deintializers with [object release] calls and a little work. Additionally I haven’t discussed how to implement super and/or dynamic dispatch but this is the core of what happens. For that things get a little more comple

I’m going to discuss how to solve dynamic dispatch first because it’ll lend its way to implementing super and subclassing support. You might be thinking we should try attempting static dispatch however you’re either going to:

  1. be writing in your target language and therefore you won’t have overhead
  2. be writing in the source language and therefore it doesn’t have knowledge of your subclass

An example is the presentViewController method. It’ll take a UIViewController not a subclass so we should just bind it. The problem is Objective-C and C++ can have rather complex implementations of things such as dynamic dispatch and vtable calls which is why we need to write some code to avoid going through that mess.

Let’s start with a basic example:

The dynamic(static) annotation may seem like an oxymoron but it says that the class uses static inheritance this will be important later on.

So in order to do this we will need to create an abstraction layer that lets us handle dynamic dispatch in both VSL and Objective-C. The structure for MyViewController along with its vtable would therefore look like (psuedo-C):

Using this we can create an ‘intermediate’ UIViewController that can both 1) store the VSL fields 2) be able to map to a clang type. Let’s see how a VSL to clang mapping would work:

Here we create the intermediate ‘VSLViewController’ in which we manage a vtable with the data which we keep as a void* since we don’t know what type that exactly is at compile-time. However what we do know is that the vtable is the first node in the data struct.

Now to see how a Clang to VSL mapping looks. Let’s introduce an extension of MyViewController.type with an ‘owner’

This is still compatible with the top but now we have an owner which represents the UIViewController, we’ll then want to ensure this is in sync with the VSL component:

…and this should be all the wrapping that is needed. myViewController[8] providing the interface for Clang. This is where the @dynamic(static) comes in, in VSL this specifies that we’ll be using the object itself’s owner field as the initial super . We’re using __attribute__((packed, aligned(8))) so we’ll be able to ensure that the location of the owner is consistent.

For summary, here’s a diagram of what is happening:

Image for post
Image for post
memory and method resolution diagram, leftwards is lower memory space

An implementation of this (along with dynamic dispatch, subclassing, super, etc. support) can be found in vsl-objc-bindgen (I’ve used the tool’s compile function earlier). You can run it using:

$ vsl-objc-bindgen gen ios iphonesimulator $(xcrun --sdk iphonesimulator --show-sdk-path)/System/Library/Frameworks/UIKit.framework/Headers/UIKit.h UIKit.m UIKit.vsl -i "<UIKit/UIKit.h>" -s UIView 
$ vsl-objc-bindgen compile ios iphonesimulator UIKit.*.m

and it’ll generate an LLVM BC file with bindings. While this targets VSL, using the above you can use the Clang API to iterate over given framework and generate code in your language of choice

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store