Wrapping Unsafe C Libraries in Rust
A practical guide to FFI without getting your knuckles bloody (part 2 of 2)
In part 1, we explored how to take a C library and write a crate of unsafe Rust bindings for it. Using this crate allows direct access to functions and other symbols in the library via unsafe, but having to use unsafe blocks everywhere in application code is not very ergonomic. We need some way to wall off that insanity.
In this article, we will explore how to wrap those functions and make them safe for normal use. We’ll go over how to define a wrapper struct that handles initialization and cleanup, and describe some traits that describe how application developers can safely use your library with threads. We’ll also talk a bit about how to turn a function’s random integer return into an ergonomic, type-checked Result, how to translate strings and arrays to and from the world of C, and how to turn raw pointers returned from C into scoped objects with inherited lifetimes.
The overall goal of this step is to dig into the C library’s documentation and make each function’s internal assumptions explicit. For example, the documentation might say that a function returns a pointer to an internal read-only data structure. Is that pointer valid for the life of the program? Or is it scope-limited to some initialized context? At the end, we will have a set of wrappers that makes these assumptions visible to Rust. Then we’ll be able to lean on Rust’s borrow checker and forced error handling to make sure everyone uses the library correctly, even from within multiple threads.
Adding enumerations
In the previous post, we discussed an example function that took an enumerated type as an argument. Here’s the cleaned up bindings.rs code:
pub const FOO_ANIMAL_UNDEFINED: u8 = 0;
pub const FOO_ANIMAL_WALRUS: u8 = 1;
pub const FOO_ANIMAL_DROP_BEAR: u8 = 2;extern "C" {
/// Argument should be one of FOO_ANIMAL_XXX
pub fn feed(animal: u8);
}
Wait a sec. The function signature says it takes a u8, but the documentation says the parameter should only be one of FOO_ANIMAL_XXX (let’s assume for the sake of our own sanity that it can safely handle the Undefined case). If we let our safe code run with arbitrary u8 as input that’s not only confusing but also potentially dangerous. Sounds like our safe wrapper should take an Animal enum and convert it.
We could write this enumeration by hand. But let’s use the enum_primitive crate to give us some extra flexibility. (I’ve omitted the rustdoc strings for brevity, though you should include them in real code):
use enum_primitive::*;enum_from_primitive! {
#[derive(Debug, Copy, Clone, PartialEq)]
#[repr(u8)]
pub enum Animal {
Undefined = FOO_ANIMAL_UNDEFINED,
Walrus = FOO_ANIMAL_WALRUS,
DropBear = FOO_ANIMAL_DROP_BEAR,
}
}
Because we tagged the struct as “representable as a u8” a simple cast is sufficient to convert an Animal into a u8. Now we can write our safe wrapper like so:
pub fn feed(animal: Animal) {
unsafe { libfoo_sys::feed(animal as u8) }
}The enum_primitive crate also gives us some helpful boilerplate functions for converting a u8 to an Option<Animal>. This might be necessary if a function returns a u8 value that actually should be treated as an enumerated type. There’s a catch: the conversion from a numeric type can fail if the number supplied doesn’t match an enumeration value. It’s up to you whether your code unwraps immediately and panics, replaces the None with a default value (“Unknown” can be used if it’s there, or added if it’s not), or simply returns the Option and lets the caller deal with it.
Initializers that return pointers
Rather than pull examples out of thin air, for the next few examples I’m going to use a rather well-known and super gnarly library as an example: OpenSSL. (Please don’t implement bindings yourself for OpenSSL; someone has already done it, and better. This is just a hopefully familiar example.)
Before you can encrypt or decrypt data with OpenSSL, you first need to call a function that allocates and initializes some context. In our example this initialization is performed by a call to SSL_CTX_new. Each function that does any work takes a pointer to this context. When we’re done using this context things need to be cleaned up and the context data needs to be destroyed using SSL_CTX_free.
We’re going to create a struct to wrap this context’s lifetime. We’ll add a function called new that does the initialization for us and returns this struct. All of the C library functions that would require the context pointer will be wrapped as Rust functions taking &self and implemented on the struct. Finally, when our struct falls out of scope we want Rust to automatically clean up for us. Hopefully this should be a familiar software pattern: it’s RAII.
Our example might look something like this:
use failure::{bail, Error};
use openssl_sys as ffi;pub struct OpenSSL {
// This pointer must never be allowed to leave the struct
ctx: *mut ffi::SSL_CTX,
}impl OpenSSL {
pub fn new() -> Result<Self, Error> {
let method = unsafe { ffi::TLS_method() };
// Manually handle null pointer returns
if method.is_null() {
bail!("TLS_method() failed");
} let ctx = unsafe { ffi::SSL_CTX_new(method) };
// Manually handle null pointer returns here
if ctx.is_null() {
bail!("SSL_CTX_new() failed");
} Ok(OpenSSL { ctx })
}
}
I’m namespacing the C library calls behind ffi to make it a bit more clear what we’re importing versus what we’re defining in the wrapper. I’m also cheating a bit and using bail from the failure crate — in real code you’d want to define an error type and use it. And yes, it looks a bit gross because we don’t have the niceties of unwrapping Option types from our returns. We have to manually check everything.
Remember: wrapping unsafe functions implies you’re doing the hard work of validating null pointers and checking for errors. This is exactly the sort of thing your wrapper MUST handle correctly. A panic early on is far better than silently passing around null or invalid pointers. We also can’t allow the ctx pointer to be copied out of the struct, because we can only guarantee it will be valid while our struct still exists.
Destructors via impl Drop
The other end is cleanup. Destructors in Rust are handled via the Drop trait. We can implement Drop for our struct so that Rust properly destroys the handle for us:
impl Drop for OpenSSL {
fn drop(&mut self) {
unsafe { ffi::SSL_CTX_free(self.ctx) }
}
}Rust also prevents drop from being called directly or being invoked twice, so you don’t have to play tricks like manually nulling out ctx after freeing it. Also, unlike C++ the destructor won’t ever get invisibly called because of invisible copies being created and deleted.
Send and Sync
So now you have a struct that contains a pointer element. But by default Rust will put some restrictions on how your struct can be used in a threaded context. Why does the language do this, and why does this matter?
By default, Rust assumes raw pointers cannot be moved between threads (!Send) and cannot be shared among threads (!Sync). And because your struct contains a raw pointer, transitively it’s neither Send nor Sync. This conservative assumption helps keep external C code from stomping all over those lovely thread safety guarantees that Rust gives us.
If your object isn’t Send, you’re very restricted in what you can do with it in a threaded program — there’s no way to even wrap it in a Mutex and pass references between threads. But maybe external documentation or clever inspection of the source code indicates that the returned context pointer is safe to move between threads. It may also indicate whether functions using this context pointer are safe to use in threaded contexts — i.e. the functions themselves are thread-safe. Rust isn’t able to make these determinations for you, because it can’t see what your library does with these pointers.
If you can make the assertion that every single use of your (internally private!) pointer obeys either of these rules, you can flat out tell Rust so. Correctly making this kind of assertion is difficult if not dangerous, and to instill in you the appropriate amount of fear Rust requires you to use the unsafe keyword.
unsafe impl Send for MyStruct {}
unsafe impl Sync for MyStruct {}Assuming you don’t allow outside access to the pointer somehow (via accessor methods or by marking the struct member pub) then you are probably safe to do the following if you can make these assertions:
- You can mark your struct
Sendif the C code dereferencing the pointer never uses thread-local storage or thread-local locking. This happens to be true for many libraries. - You can mark your struct
Syncif all C code able to dereference the pointer always dereferences in a thread-safe manner, i.e. consistent with safe Rust. Most libraries that obey this rule will tell you so in the documentation, and they internally guard every library call with a mutex.
Functions that return pointers
So let’s assume we’ve got our struct set up with new and drop implementations. We’re happily churning through the list of functions that take this context pointer, and for each one we want to expose we’re implementing a safe version against our struct that takes &self. Then we run into something like this (fictional for simplicity, but not far off):
// Always returns valid data, never fails
SSL_CIPHER *SSL_CTX_get_cipher(const SSL_CTX *ctx);We obviously don’t want to return raw pointers from our wrapper, that’s not very ergonomic. The whole point of this is to make sure library users don’t have to use unsafe.
After reading the documentation we discover that SSL_CIPHER is a struct, and the pointer returned is valid as long as our SSL_CTX isn’t freed. Hey, that kinda sounds like a lifetime bound. So our first approach might look like this:
pub fn get_cipher(&self) -> &ffi::SSL_CIPHER {
unsafe {
let cipher = ffi::SSL_CTX_get_cipher(self.ctx);
// Dereference the pointer, then turn it into a reference.
// Remember: derefing a pointer is unsafe!
&*cipher
}
}Dereferencing and then immediately taking the address of a pointer creates what’s called an unbounded lifetime. This isn’t what we want, so we immediately constrain the lifetime via the return type. We don’t explicitly specify a lifetime, but let’s recall the rules for lifetime elision from the Rust handbook. The lifetime of the return value, in this case, will be constrained by default to be the same as the lifetime for &self. That’s a sane bound, so this implementation looks safe.
But we can go further. That SSL_CIPHER is typically used as a context pointer with its own associated functions. As is, having our safe code return a reference to a C struct isn’t ergonomic at all. What we want to return is a Rust struct with its own associated behavior matching the C library. But we also should retain the lifetime association: “This cipher object is only valid as long as the OpenSSL object you got it from is still alive.”
So let’s assume we go through the work of creating a Cipher struct to wrap that pointer, and we want to tell Rust that the struct has some sort of lifetime that depends on our OpenSSL object:
pub fn get_cipher<'a>(&'a self) -> Cipher<'a> {
unsafe {
let cipher = ffi::SSL_CTX_get_cipher(self.ctx);
Cipher::from(&self, cipher)
}
}// Something is missing here...
pub struct Cipher<'a> {
cipher: *const ffi::SSL_CIPHER,
}fn from<'a>(_: &'a OpenSSL, cipher: *const ffi::SSL_CIPHER)
-> Cipher<'a> {
Cipher { cipher }
}
Unfortunately this won’t compile, because Rust says “hey, you declared a lifetime associated with your struct, but it’s not used anywhere!” So we need to declare somehow that yes, the internals depend on a reference we can’t immediately see.
use std::marker::PhantomData;pub struct Cipher<'a> {
cipher: *const ffi::SSL_CIPHER,
phantom: PhantomData<&'a OpenSSL>,
}fn from<'a>(_: &'a OpenSSL, cipher: *const ffi::SSL_CIPHER)
-> Cipher<'a> {
Cipher { cipher, phantom: PhantomData }
}
You can think of this as saying to the compiler, “Treat this struct as if it contains a reference to an OpenSSL, with lifetime 'a”. Where does that lifetime come from? We provide it when we call our from with a reference &self.
PhantomData doesn’t actually take up any space, and it disappears in compiled code. But it allows the compiler to reason about lifetime correctness. Now our wrapper users can’t accidentally hold onto a Cipher after freeing its parent OpenSSL.
Functions that may return errors
Consider the following C function:
int foo_get_widget(const foo_ctx_t*, widget_struct*);We’re expected to pass in a pointer, which the function will fill in. If this function returns 0 everything is fine, and we can trust the output was populated correctly. Otherwise, we need to return an error.
It’s far more ergonomic to return an owned struct with the data rather than demand that Rust callers create a mutable struct and pass a mut reference (though you can provide both if it makes sense to do so).
In the following examples I’m assuming the custom error type is defined elsewhere, and allows conversion from the appropriate types.
use std::mem::MaybeUninit;pub fn get_widget(&self) -> Result<widget_struct, GetError> {
let mut widget = MaybeUninit::uninit();
unsafe {
match foo_get_widget(self.context, widget.as_mut_ptr()) {
0 => Ok(widget.assume_init()),
x => Err(GetError::from(x)),
}
}
}
Ed: thanks to reddit /u/Cocalus for pointing out that mem::uninitialized() is deprecated. Hopefully I got the fix right!
Above, widget_struct doesn’t implement Default because neither a “default” nor zeroed-out constructor makes sense. Instead we tell Rust to leave the struct memory uninitialized, and we therefore assert that our external function is responsible for correctly initializing every field of the struct.
Some functions don’t return anything useful, but still can error out.
int foo_update(const foo_ctx_t*);You might be tempted to just cast the integer value over to your enumerated type and be done with it. Don’t do that! Correctly written Rust code will return a Result when things can fail, and so should you. But what about the “success” value? For this we should use () to tell the caller there’s no returned data. But the caller will still need to unwrap or otherwise handle errors.
update(&self) -> Result<(), UpdateError> {
match unsafe { foo_update(self.context) } {
0 => Ok(()),
x => Err(UpdateError::from(x)),
}
}Strings in FFI
Rust doesn’t store strings as null-terminated buffers of char as C does; internally it stores a buffer and a length. Because the types don’t exactly line up, this means there’s some trickery required to move from the world of Rust strings to C char arrays and back. Thankfully there are some built-in types to help manage this, but they come with a lot of strings attached. Some conversions allocate (because they need to either alter the string representation or add a terminating null) and some do not. Sometimes it’s safe to get away with not performing a copy, and sometimes it’s not. The documentation does explain which is which, but there’s a lot to read through. Here’s the executive summary.
If you’re somehow producing a Rust String and need to pass it as a temporary const* c_char to C code (so it can make a copy of the string for its own use), you can convert it to CString and then call as_ptr. If your wrapper signature borrows a &str, first convert to &CStr and then invoke as_ptr. The pointer in both cases is only valid as long as the Rust reference would normally be valid. Raw pointers strip the borrow checker safeties off and require us to maintain this invariant ourselves. If you screw this up, Rust can’t help you.
For cases where you’re getting a const char * from a C function and want to convert it to something Rust can use, you’ll need to ensure it’s not null, then convert it into a &CStr with an appropriate lifetime. If you can’t figure out how to express an appropriate lifetime, the safest thing is to immediately convert it into an owned String!
Some other things to note:
CString::as_ptrhas a footgun that makes it difficult to use correctly. Read the section marked WARNING in the documentation, and make sure yourCStringremains in scope until your C code returns.- Rust strings can legally have a null
\0byte in the middle, but C strings can’t (because it would terminate the string). For this reason, trying to convert a&strinto a&CStror aStringinto aCStringcan potentially fail. Your code needs to handle this. - Once you’ve converted a raw C pointer into a
&CStryou still have to perform (or unsafely skip) some validation before using it as&strin native safe Rust. C uses arbitrary string encodings, while Rust strings are always UTF-8. Most C libraries these days return valid UTF-8 strings anyway (ASCII is a subset, so even legacy apps are probably OK). Functions allowed to allocate (likeCStr::to_str_lossy) will replace invalid characters with the UTF “replacement character” if necessary, while others likeCStr::to_strwill simply return aResult<&str, _>. Read the documentation, and choose the correct function for your needs. - If your library returns paths, use
OsStringand&OsStrin your wrapper instead ofStringand&str.
Arrays and lengths
If the library accepts a pointer to T and a length, it’s straightforward to implement a wrapper that takes a slice and breaks it into a pointer and length. But what about the other way around? Turns out there’s a library function for that, too. Remember to check for null, make sure the size is the number of elements, and double check that the return lifetime is correct. Again, if you can’t guarantee lifetime correctness just return an owned collection such as a Vec instead.
Callbacks
Callback signatures typically get generated as Option<unsafe extern "C" fn ...> in bindgen. So when you write your callbacks in Rust, you obviously need to decorate them with unsafe extern "C" (they don’t need to be pub), and then when you pass them to the C library you just wrap the name in Some. Pretty straightforward.
The catch is that panic unwinding into C code is… well, let’s just say it’s bad. And in theory panics can happen pretty much anywhere in Rust code. So to be safe, we need to wrap our callback body inside catch_unwind. Normally it’s not smart or sane to catch panics, but this is the exception. No pun intended.
unsafe extern "C" fn foo_fn_cb_wrapper() {
if let Err(e) = catch_unwind(|| {
// callback body goes here
}) {
// Code here must be panic-free.
// Sane things to do:
// log failure and/or kill the program
eprintln!("{:?}", e);
// Abort is safe because it doesn't unwind.
std::process::abort();
}
}Common patterns
As you’re writing these wrappers, you’ll probably come across some patterns that can be easily expressed in either functions or macros. For example, library functions might always return an int, which always represents the same set of nonzero errors. Writing a private macro that calls the unsafe code and casts the return into Result<(), LibraryError> can save a lot of boilerplate. Keep an eye out for these constructs, and with a little refactoring you can save yourself hours of effort.
I’m not going to lie. Doing this correctly is a lot of work. But doing it right results in code that is eerily bug-free. If your C library is solid and your wrapper layer is correct, you won’t see a single segmentation fault or buffer overflow. You’ll see any errors immediately. When you make what used to be pointer mistakes in your application code it simply won’t compile; when application code compiles, it will just work.
