Calling C# natively from Rust.

…or how CMake drove me to create an eldritch monstrosity.

A couple of months ago, I created my first Rust program; a music manager called seiri.

Yes, it’s an electron app, but most of the heavy lifting is Rust.

seiri has four main components that work in tandem. The user interface, which is written as an electron application, the query engine that allows the music library to be searched extremely effectively, the tag library, which reads the music metadata tags from a variety of files, and a folder watcher, which is exactly what it sounds like.

The query engine is in fact a full parser and lexer that transpiles the query language into SQL statements, and using a crate called neon allows the user interface to query the database almost instantly. But this is not a story about Rust and Javascript, this is a story about Rust and C#.

seiri is actually a rewrite of a previous, much buggier program that I used to organize my music that was written in C#. The tag library of choice was of course, taglib-sharp, a port of the C++ library TagLib to the .NET ecosystem. Since Rust unfortunately doesn’t have its own native port of TagLib, and any C bindings available didn’t expose the picture API, the most obvious thing to do was to use the C# library with Rust somehow, right?

My first instinct was passing JSON through stdin, as one is wont to do when doing cross language interop. That was unacceptably slow. Protobuf was another option, but before that went anywhere, I remembered reading about an experimental Microsoft technology called .NET Native. After doing some digging, I got extremely excited when I found dotnet/corert, and discovered that it recently gained the ability to build native, static libraries of C# code. Not CIL bytecode, no JIT compiler included, but bona fide, linkable, bare metal machine code, for C#, a safe, garbage collected language.

Preparing C# for Native Interop

After setting up the build dependencies and creating a C# project, in order to use CoreRT for building native libraries, you will need to specify that your project is a static library.

Under the csproj project file, specify the output type as Library, a netcoreapp2.1 target framework, and the type of native library. To link properly with Rust, we’ll be directing CoreRT to produce a Static library.

<PropertyGroup>
    <OutputType>Library</OutputType>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <RuntimeIdentifiers>win-x64;osx-x64;linux-x64</RuntimeIdentifiers>
    <NativeLib>Static</NativeLib>
  </PropertyGroup>

Then, you will need to add the ILCompiler NuGet package to your project. In the root of your project, run the following command to generate a nuget.config file.

dotnet new nuget

Under the <clear/> item in <packageSources> , add the following MyGet feeds.

<add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />

Finally, add the ILCompiler package using the dotnet CLI.

dotnet add package Microsoft.DotNet.ILCompiler -v 1.0.0-alpha-*

Your C# project is now set up to build as a static library.

Exposing C# methods to the C FFI

In order to interop with Rust, we need to expose some C# functions through the C FFI. Create a class called NativeCallableAttribute with the following members.

namespace System.Runtime.InteropServices
{
    [AttributeUsage(AttributeTargets.Method)]
    public sealed class NativeCallableAttribute : Attribute
    {
        public string EntryPoint;
        public CallingConvention CallingConvention;
        public NativeCallableAttribute() { }
    }
}

Creating a Lib C# class, lets get a hello world going.

[NativeCallable(EntryPoint = "add_dotnet", CallingConvention = CallingConvention.Cdecl)]
public static int Add(int a, int b) {
      return a + b;
}

Build your C# library with the following command in your project root, and copy it to a lib folder next to your Rust code.

dotnet publish /t:LinkNative /p:NativeLib=Static -c Release -r win-x64

Change the -r option to either linux-x64 or osx-x64 respectively on Linux or macOS platforms.

Finally, on the Rust side…

#[link(name = "bootstrapperdll", kind = "static")] 
#[link(name = "Runtime", kind = "static")]
#[link(name = "mycsharplibrary", kind = "static")]
extern "C" {
    pub fn add_dotnet(a: i32, b: i32) -> i32;
}
let result = add_dotnet(1, 2);
println!("{}", result);

You will need to link against bootstrapperdll , which bootstraps the CLR garbage collector, Runtime which is the CLR runtime library, and finally your C# static library.

In your build.rs build script, tell cargo where to look for your library and the bootstrap libraries.

println!("cargo:rustc-link-search=native={}", "./lib");
println!("cargo:rustc-link-search=native={}", "/path/to/bootstrapperdll");

The bootstrapper and runtime libraries are in the $HOME/.nuget/packages/runtime.[runtime identifier].microsoft.dotnet.ilcompiler/[version]/sdk folder, so find that path and copy it in place.

If nothing goes wrong, you should get a result after you run cargo run

Returning structs from C#

You are not limited to primitives when passing values through the FFI barrier. C# can return C-compatible structs that are readable from Rust. However, the struct must only contain primitive types and IntPtr

[StructLayout(LayoutKind.Sequential)]
public struct MyStruct 
{
...
}

On Rust, this proceeds as if you were calling a C FFI function.

#[repr(C)]
pub struct MyStruct {
    pub ...,
}

Strings

Strings in C# are not C compatible and must be manually marshalled into an IntPtr, like so.

[NativeCallable(EntryPoint = "hello_world", CallingConvention = CallingConvention.Cdecl)]
public IntPtr HelloWorld() 
{
   return Marshal.StringToCoTaskMemUTF8("Hello World");
}

This allocates unmanaged memory for a null-terminated UTF8 C compatible char* string, and returns a pointer to that memory. On Rust, you can treat this as a *const c_char , and deal with it as you would a normal C string. However, you can not free this string with libc::free . Instead, expose the CLR free method like so:

[NativeCallable(EntryPoint = "free_corert", CallingConvention = CallingConvention.Cdecl)]
public static void Free(IntPtr ptr) {
     Marshal.FreeCoTaskMem(ptr);
}

And call free_corert(ptr: *mut c_void)in Rust to free memory once you’re finished with the string.

Reflection, or more appropriately RTTI

C# has a powerful reflection API that is often used by many libraries. Since the .NET compiler will now emit native code, you need to manually specify which namespaces to include runtime type information (RTTI) for. Unlike C++, this isn’t done automatically, and if you fail to specify the correct namespaces, reflection will fail spectacularly. Fortunately, it’s not that difficult.

First, under the root folder of your C# project, create rd.xml with the following contents.

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata"> 
    <Application>
        <Assembly Name="[name of your assembly]" Dynamic="Required All"/>             
     </Application> 
</Directives>

If you use any C# libraries that use reflection, such as Newtonsoft.Json , add an entry for those libraries as well, like so

<Assembly Name="Newtonsoft.Json" Dynamic="Required All"/>

Finally, include rd.xml as a resource in your csproj

<ItemGroup>
    <RdXmlFile Include="rd.xml" />
    <!--Include stack trace data -->
    <IlcArg Include="--stacktracedata" /> 
</ItemGroup>

Putting it all together

Without a proper build script, compiling C# and Rust separately is extremely unwieldy. The build script will need to discover the .NET Core and ILCompiler SDK versions, call the .NET compiler, and direct cargo to link to those libraries. See how seiri wraps everything up with this build script. I’ve had a redditor approach me in adapting this build script to something more generic, so if that pans out, that would make things much easier to get started.

Conclusion

It works? I was surprised and shocked as you may or may not be that it was possible for such an abomination to exist. Surprisingly, the biggest drawback was a bloated executable; having to link not only the entire .NET Runtime Library as well as the CLR garbage collector resulted in an executable size of about 16 megabytes. Not only did it just work, it was reasonably performant, on par or even better than normal, JITted C#. It was really the best of both worlds.

However, when I last messed with this, anything more complex than returning primitives would completely blow up during the linking step on Linux, no matter what I tried. The redditor that contacted me about generalizing the build script managed to get everything working on Linux though, so YMMV. Theoretically there is nothing stopping Linux compatibility, but I suspect that there may be a conflict with the version of clang Rust uses, and the version of clang that CoreRT uses, but that may have been fixed by now.

As for seiri, I’ve only really written all this up for posterity. The main barrier that prevented me from using the C++ version of TagLib was the horrible, horrible, perhaps even more Lovecraftian than this unholy matrimony between garbage collectors and borrow checkers, build system that is CMake. It must mean something that trying to get TagLib’s CMake build files to cooperate with me drove me to instead give up, and use an experimental, alpha technology instead.

All good things must come to an end, and as much as it saddens me, after dipping my toes in some C++ hellfire and coming back to this project in a few months, I’ve finally figured out how to get CMake and the C++ version of TagLib to work with cargo’s build system and am working on replacing taglib-sharp with TagLib proper, for proper Linux support if nothing else at all. This blog post serves as a reminder and as an instructional for anyone willing to go as far as I did to avoid CMake and C++ build systems that they’d be willing to create their own Frankenstein’s Monster.

Final Remarks

I’ve noticed that this blog post is now linked as an example on the CoreRT docs as a reference. I’ve since switched seiri over to a pure C++ approach, but for anyone looking for a real life example, this would be the tree where I still used C# in seiri. This includes a build script (that works only on Windows) that will link required libraries automatically, but may be outdated as of time of writing.