Executing Rust in Python using WebAssembly 🐍🦀

The snake and the crab

Using PythonMonkey and wasm-bindgen to call Rust from Python!

Youtube video version of this article

Introduction

This tutorial will go over using wasm-bindgen and PythonMonkey to compile and execute rust code from Python. Wasm-bindgen is a Rust library that makes it easier to call Rust WASM code from JavaScript and vice versa. PythonMonkey is a Python library for running JavaScript and WebAssembly in Python. Using PythonMonkey and wasm-bindgen, we can easily call Rust code from Python in a secure and portable WebAssembly sandbox.

This tutorial is based on an MDN article titled Compiling from Rust to WebAssembly.

All the code from article will be available on GitHub

Required Software Installation

Install Rust, and Cargo:

Rust uses Cargo for package management. Follow the instructions here: https://doc.rust-lang.org/cargo/getting-started/installation.html

Install Python and pip

Install Python with a minimum version of 3.8.

Install PythonMonkey

PythonMonkey is required to run WebAssembly from Python.

pip install pythonmonkey

Install wasm-pack

wasm-pack is a command line tool for compiling rust code to WASM and providing a JavaScript package as an API for the code.

cargo install wasm-pack

Building a Rust WebAssembly Package

First, use cargo to generate a new library for us to get started in.

cargo new --lib my-rust-library

This creates a new library in a sub-directory named my-rust-library with everything you need to get going:

├── Cargo.toml
└── src
    └── lib.rs

First, we have Cargo.toml; this is the file that we use to configure our build. If you’ve used Gemfile from Bundler or package.json from npm, this is likely to be familiar; Cargo works in a similar manner to both of them.

Next, Cargo has generated some Rust code for us in src/lib.rs:

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

We won’t use this test code at all, so go ahead and delete it.

Writing a Rust Library for our Python to Call

Let’s put this code into src/lib.rs instead:

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn add(a: u32, b: u32) -> u32{
  a + b
}

add is a simple function that returns the sum of two numbers. More complicated functions can be used, but let’s start with a simple example for now.

Compiling our code to WebAssembly

To compile our code correctly, we first need to configure it with Cargo.toml. Open this file, and change its contents to look like this:

[package]
name = "hello-wasm"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
wasm-bindgen = { version = "0.2.84" }

Building the package

Now that we’ve got everything set up, let’s build the package. Type this into your terminal:

wasm-pack build --target nodejs

This command performs a number of actions:

• Compiles our Rust code to WebAssembly.
• Runs wasm-bindgen on that WebAssembly, generating a JavaScript file that wraps up that WebAssembly file into a module that can be loaded from JavaScript.
• Creates a pkg directory and moves that JavaScript file and your WebAssembly code into it.

The main outcome is that we now have a JavaScript package in the pkg/ directory

An Extra Build Step

Currently, PythonMonkey has limited support for Node.js libraries, so we’ll need to add an extra step to the build process to convert the calls to Node.js libraries with PythonMonkey friendly ones.

The call to wasm-pack outputs code using readFileSync from Node.js’ fs library and join from its path library. We’ll need to replace these with the following:

function readFileSync(filename) {
  python.exec(`
def getBytes(filename):
  with open(filename, 'rb') as f:
    return bytearray(f.read())
  `);
  return python.eval('getBytes')(filename)
}

[a,b].join('/') # require('path').join(a,b)

To simplify and automate this process, I’ll include these in a shell script called build.sh that runs wasm-pack and injects these function definitions using sed.

Place build.sh in the same directory as your Cargo.toml and paste in the following code:

#!/usr/bin/env bash
#
# This shell script compiles the rust library to WebAssembly and creates a JS package using wasm-pack. 
# It also replaces any Node.js specific code with PythonMonkey code using sed.
#
# Note: this script is only compatible with unix.
#

# compiles rust code to wasm and creates a package targetting Node.js
wasm-pack build --target nodejs

# replaces calls to require('path') and require('fs') with pythonmonkey non Node.js equivalents
NEW_READ_FILE_SYNC="function pyReadFileSync(filename) {\n  python.exec(\n\\\`\ndef getBytes(filename):\n  file = open(filename, 'rb')\n  return bytearray(file.read())\n\\\`\n  );\n  return python.eval('getBytes')(filename)\n}\n"

find ./pkg/*js -type f -exec sed -i "s/require('path').join(\(.*\));/[\1]\.join('\/');/g" {} \;
find ./pkg/*js -type f -exec sed -i "s/require('fs').readFileSync(\(.*\));/(${NEW_READ_FILE_SYNC})(\1)/g" {} \;

Now, all we have to do is run ./build.sh and it will compile our Rust to WASM and replace Node.js specific code with Python and vanilla JS equivalents.

Executing the Rust code from Python

Run your new build.sh script to compile your Rust code and get started with calling WASM code from Python!

bash ./buils.sh

Here is a simple python script that calls our add function:

import pythonmonkey as pm

test = pm.require("./pkg/hello_wasm");

print(test.add(1,2))

Run this script with python3 main.py and it will call our add function defined in rust and output 3.0 !

Conclusion

Calling Rust code compiled to WASM in Python using PythonMonkey

Well we wrote rust, compiled it to WASM and ran it in Python!

Although this example is very simple, it can be expanded on to run more complicated Rust code from Python. One use case could be a Python library that uses Rust for extra efficiency for complex operations.

Check out more about PythonMonkey here: https://pythonmonkey.io and consider giving it a star on GitHub if you like the project https://github.com/Distributive-Network/PythonMonkey/ 🐍🐵.

Refer to PythonMonkey’s documentation for more information: https://docs.pythonmonkey.io/ .