Complex types with Rust’s FFI
Interop with object methods, structs, and arrays
When I wrote about calling Rust functions from Unity3D , it was my first time working with a foreign function interface (FFI) and there was a lot I didn’t understand beyond calling simple functions with primitives.
How do I call methods? How do I pass arrays? How do I pass structs back and forth? Here’s what I’ve come up with…
Note that all of the examples below use Node.js. The principles are the same in Unity3D, C#, and other languages.
A quick note about usize
Often, marshaling between types is pretty straightforward: f64 to double, u64 to ulong, or simply i32 to int. Rust’s usize, however, turned out to be the most varied, and most ambiguous, type-mapping amongst host languages. The usize type represents an unsigned number the width of a pointer (like 32-bit or 64-bit). This varies by the host platform’s OS so while you could use a ulong or uint32 on your machine it might break elsewhere. Since Rust uses usize quite often for ranges and indices: always make sure to use a type that represents a platform-specific width. In Node.js you’ll want size_t and in C# (or Unity3D) UIntPtr seems to do the trick.
Working with methods
Since we’re effectively passing memory references around, the notion of an object with methods doesn’t really exist across the FFI boundary. To work around this limitation, we can define static functions that operates on pointers that we reinterpret as the original object. The host then holds this pointer and uses it when calling these functions.
Here’s a simple counter struct with increment and decrement methods that we’ll use as the basis for our examples:
Now let’s add our FFI. At a minimum we need to provide:
- A constructor — the constructor instantiates an object in memory and returns a pointer to it.
- A destructor for the instantiated objects. We’re responsible for cleaning up memory allocated by the foreign language.
- A function to act as a proxy for each method on the object that we want to call.
Here’s what that looks like:
We utilize Rust for memory allocation to create our counter on the heap, using Box, then transmute this box into a raw pointer. This trickery avoids having to manually allocate the memory and seems to be the most canonical way to allocate the counter. Our destructor works similarly by transmuting the counter’s pointer back into a Box then letting it automatically drop.
Finally, each function acting as a proxy takes a pointer as its first argument. The function converts this pointer to the original type and calls the desired method passing through any arguments, and finally returning the result (if any). Unlike our destructor, we don’t want to transmute back these pointers into a box until we’re ready to destroy it.
Calling the FFI is pretty straightforward, relying on the host language’s pointer type:
Working with structs
Sometimes functions may require a number of arguments. To avoid a complicated function signature, we can use configuration structs to group related arguments. Structs work well for this task because they can be described linearly in memory with a flat structure (matching the C struct definition) so passing a struct in and out of Rust is pretty straightforward. Classes, on the other hand, involve more indirection and, therefore, cannot be easily passed.
The main concern for the host language is the memory layout of the struct properties. Dynamic languages like Node.js provide tools for defining structs with the appropriate layout. In C# you can use the StructLayout attribute with LayoutKind.Sequential.
In this example, the counter is modified to accept a configuration struct containing the initial value and the amount to increment and decrement by:
With the FFI, Rust handles the struct conversion directly so we don’t need to do anything special:
In Node, we define a matching struct type for Args and use it in our interface specification:
Working with arrays
Passing an array turns out to be the least straightforward of the three techniques since we cannot simply pass the array back and forth like we can with pointers or structs. An array can most generally be represented by a pointer to the first element in the array and a length so that’s what we’ll use.
Another issue is ownership: who owns the array’s memory? The safest option is to let the host be responsible for the memory since it has the most information about how the memory should be freed. You pass an array in, manipulate it in place and then, instead of returning the array, the caller can simply read its contents when the function is complete.
The array type in Rust must have a known length at compile time so we need to use a slice, or a “view” into an array, which we’ll sum into our counter:
In the FFI we need a pointer to the first value in the slice and its length. Then we can use std::slice::from_raw_parts to reassemble the slice (or std::vec::Vec::from_raw_parts to create a vector).
From the host language we can simply specify an array type as the argument:
A better interface
To make things even cleaner, let’s wrap up our host FFI into a class that exposes a more natural interface. Most importantly we can hide the use of the pointer since the caller should not need to worry about it (and misuse of the pointer can cause errors or unexpected behavior).
Conclusion
And that’s it! You can play around with the code samples on Github.
Below are some of the resources I used when researching how to do the things in this post.
If I got anything wrong or if you have any questions please let me know via Twitter or email.
