Open in app

Sign in

Write

Sign in

Calling Rust functions from Deno — Part 2 Passing buffers

Mayank C
Tech Tonic

--

Introduction

Deno supports loading & calling of native functions provided by a shared library (mostly written in C, C++, or Rust). The standardized name for this mechanism is: FFI (Foreign Function Interface).

The definition of FFI is (source Wikipedia):

A foreign function interface (FFI) is a mechanism by which a program written in one programming language can call routines or make use of services written in another. The primary function of a foreign function interface is to mate the semantics and calling conventions of one programming language (the host language, or the language which defines the FFI), with the semantics and conventions of another (the guest language)

In simple words, FFI allows calling functions provided by a different language.

The first article in this series covered passing primitive types like integers, floats, etc.

In the second article (i.e. this article), we’ll learn how to pass buffers containing string data to and from the native library.

Let’s get started.

Basics

Imports

The primary function for loading a shared library is dlOpen which is part of the Deno core runtime. Therefore, no imports are required.

Permissions

To use this feature, two command-line flags are required:

  • — allow-ffi
  • — unstable

Even though this feature is a bit old now, it is still present under unstable umbrella.

Types

The FFI supports types like void, signed/unsigned integer, floating point numbers, buffers, pointers, etc. In the first article, we’ve covered primitive types like integers. In this article, we will cover more complex types like buffers, etc.

Here is the complete list of types supported by FFI:

Native types =>      Void,
  U8,
  I8,
  U16,
  I16,
  U32,
  I32,
  U64,
  I64,
  USize,
  ISize,
  F32,
  F64,
  Pointer

DlOpen

The primary function to open a dynamic library is: dlOpen. There are two mandatory parameters:

  • filename: Path of the shared library to open
  • symbols: The symbols to import from the shared library along with the signature (input & output). For input, an array of parameters needs to be specified (it could be empty). For output, a single result needs to be specified (it could be void too).
function dlopen (
    filename: string,
    symbols: S,
): DynamicLibrary<S>;

For example-

const dylib = Deno.dlopen('/var/lib/xyz.dylib', {
  "do_something": { parameters: [], result: "void" },
  "do_something_else": { parameters: ["u32", "u32"], result: "u32" },
});

In the above example, do_something & do_something_else symbols/functions are imported from the library.

Calling symbols

Once the shared library is opened & symbols are imported, they can be called anytime by optionally passing the required parameters (as per the signature).

For example-

dylib.symbols.do_something();
dylib.symbols.do_something_else(123, 456);

Closing the library

When the library’s use is completed, it can be closed by calling the close function.

dylib.close();

That’s all about the general introduction. Now, let’s write a Rust code, build it, and then call it from Deno.

Steps to call C functions

Step 1: Write a C program

The first step is to write a Rust program. Our Rust program has a single function, called to_and_fro_buffer, that takes a const u8* as input and returns a const u8*. The const u8* is typecasted to string in the Rust code and then printed using println. The functions then returns a static string. In other words, the function takes an arbitrary string buffer as the input, and returns another string buffer.

This program is specifically for mac

// lib.rsuse std::ffi::CStr;#[allow(clippy::not_unsafe_ptr_arg_deref)]
#[no_mangle]
pub extern "C" fn to_and_fro_buffer(buf: *const u8) -> *const u8 {
    let c_string = unsafe { CStr::from_ptr(buf as *const i8) };
    println!("RECEIVED IN RUST SPACE => {}", c_string.to_str().unwrap());
    return "Hello back from RUST space\0".as_bytes().as_ptr() as *const u8;
}

Step 2: Write a build file

The second step is to write a build file that can create a shared library. The build file produces a mac cdylib.

// cargo.toml[package]
name = "deno_ffi_test"
version = "0.1.0"
publish = false[lib]
crate-type = ["cdylib"]

Step 3: Build a shared library

The third step is to compile the source and build a shared library. This is done using a single command: cargo build.

As Mac is M1, the target architecture needs to be set

> cargo build --target=aarch64-apple-darwin
   Compiling deno_ffi_test v0.1.0 (/Users/mayankc/Work/source/denoExamples/denoFfiTest)
    Finished dev [unoptimized + debuginfo] target(s) in 1.09s

The output is present in the target folder:

> ls -R target/
aarch64-apple-darwin debugtarget//aarch64-apple-darwin:
CACHEDIR.TAG debugtarget//aarch64-apple-darwin/debug:
build   examples  libdeno_ffi_test.d
deps   incremental  libdeno_ffi_test.dylibtarget//aarch64-apple-darwin/debug/build:target//aarch64-apple-darwin/debug/deps:
deno_ffi_test.205vh77u030dp9ap.rcgu.o deno_ffi_test.4krka3qempjny888.rcgu.o deno_ffi_test.eboajdgv0jm14ni.rcgu.o
deno_ffi_test.39xbiigyws4d95n4.rcgu.o deno_ffi_test.4v20jrj4vi2aj9m8.rcgu.o libdeno_ffi_test.dylib
deno_ffi_test.49rohym6wwrpzpi2.rcgu.o deno_ffi_test.dtarget//aarch64-apple-darwin/debug/examples:target//aarch64-apple-darwin/debug/incremental:
deno_ffi_test-1vl1t8d9xd9thtarget//aarch64-apple-darwin/debug/incremental/deno_ffi_test-1vl1t8d9xd9th:
s-g8u6nayj68-1x00m8d-3rj1dno1r4id s-g8u6nayj68-1x00m8d.locktarget//aarch64-apple-darwin/debug/incremental/deno_ffi_test-1vl1t8d9xd9th/s-g8u6nayj68-1x00m8d-3rj1dno1r4id:
205vh77u030dp9ap.o 49rohym6wwrpzpi2.o 4v20jrj4vi2aj9m8.o eboajdgv0jm14ni.o work-products.bin
39xbiigyws4d95n4.o 4krka3qempjny888.o dep-graph.bin  query-cache.bintarget//debug:
build  deps  examples incrementaltarget//debug/build:target//debug/deps:target//debug/examples:target//debug/incremental:

We’re only interested in libdeno_ffi_test.dylib.

Step 4: Open shared library in Deno

The fourth step is to open our shared library in Deno. This step uses dlOpen function. The path to DYLIB is specified, along with the to_and_fro_buffer function that needs to be imported from the shared library. As the function takes an input and returns an output, we need to specify both in the dlOpen step so that the appropriate symbols can be imported. The buffer is specified as of type ‘pointer’ for both input and output.

const libName = "./denoFfiTest/target/aarch64-apple-darwin/debug/libdeno_ffi_test.dylib";const dylib = Deno.dlopen(libName, {
  "to_and_fro_buffer": { parameters: ["pointer"], result: "pointer" },
});

If the dlOpen function is able to open the shared library & import the symbols, it’d return a DynamicLibrary object.

Step 5: Calling the function

The fifth step is when we call the function provided by the shared library. This is exactly why we did all the work!

The to_and_fro_buffer function needs a buffer as the input and returns a buffer as the output.

  • The input buffer must be provided as a Uint8Array (like bytes)
  • The output buffer needs to be ‘viewed’ using UnsafePointerView and then the string can be obtained by calling getCString function

Here is the complete code:

const libName = "./denoFfiTest/target/aarch64-apple-darwin/debug/libdeno_ffi_test.dylib";const dylib = Deno.dlopen(libName, {
  "to_and_fro_buffer": { parameters: ["pointer"], result: "pointer" },
});const buffer = new TextEncoder().encode("Hello coming from Deno space");const ret = dylib.symbols.to_and_fro_buffer(buffer);const dataView = new Deno.UnsafePointerView(ret);
console.log("RECEIVED IN DENO SPACE => ", dataView.getCString());

Let’s run it:

> deno run --allow-ffi --unstable app.ts 
RECEIVED IN RUST SPACE => Hello coming from Deno space
RECEIVED IN DENO SPACE =>  Hello back from RUST space

The native function gets called successfully with an arbitrary buffer as the input. The same native function is able to return an arbitrary buffer data to the Deno user space. The first line of the output comes from the Rust function and the second line of the output comes from Deno space.

Step 6: Closing the library

The last step is to close the library as we’re done with it.

dylib.close();

That’s all about calling Rust functions from Deno with to and fro of arbitrary buffers.

To know about how to call C functions from Deno with arbitrary buffers, the article can be seen here.

To know about how to call C functions from Deno with primitive types, the article can be seen here.

To know about how to call rust functions from Deno with primitive types, the article can be seen here.

To know about how to call C functions from Deno with primitive types, the article can be seen here. (Windows)

To know about how to call C functions from Deno with arbitrary buffers, the article can be seen here. (Windows)

--

--

Mayank C
Tech Tonic

Written by Mayank C

9.4K Followers
Editor for

800 medium articles