Strings in WebAssembly (Wasm)

Timothy McCallum
Feb 4, 2020 · 16 min read
Image for post
Image for post

The importance of strings

Computer programs can execute successfully, using only numbers. However, in order to facilitate human-computer interaction, human readable characters and words are required. This is especially the case, when we consider how humans interact with applications on the Web. One strong example of this is the fact that humans choose to use domain names, rather than numerical IP addresses, when they visit sites on the Web.

Strings in WebAssembly (Wasm)

Update: 2020–08–03

Image for post
Image for post
Strings in WebAssembly
  • i64, a 64-bit integer (equivalent to C++’s signed long long int)
  • f32, 32-bit float (equivalent to C++’s float)
  • f64, 64-bit float (equivalent to C++’s double)

Strings in WebAssembly — how to?

Now, it is possible to turn a high-level value (such as a string) into a set of numbers. If this is achieved, then we could pass these sets of numbers (which represent strings) back and forth between our functions. However, there are couple of issue with this.

Strings in Rust

The String

A String in Rust can be thought of as a Vec<u8> that is guaranteed to hold well-formed UTF-8 (Blandy and Orendorff, 2017).

The &str

A &str in Rust is a reference to a run of UTF-8 text owned by someone else; &str is a fat pointer, containing both the address of the actual data and its length. You can think of &str as being nothing more than a &[u8] that is guaranteed to hold well-formed UTF-8 (Blandy and Orendorff, 2017).

Strings at compile time — stored in the executable

A string literal is a &str that refers to preallocated text, typically stored in read-only memory, along with the programs machine code; bytes are created when the program begins execution, and last until the program ends. It is therefore impossible to modify a &str(Blandy and Orendorff, 2017).

fn my_function(the_string: &str) -> &str {
// code ...
}

Strings at runtime — allocated and freed at runtime

New strings can be created at runtime, using String. A string literal can be converted to a String using the following methods. The to_string() and String::from do the same thing, so which you choose is a matter of style (Klabnik and Nichols, 2019).

let s = "the string literal".to_string();
let s = String::from("the string literal");

Converting strings to numbers

The following Rust code takes the string hello and converts it to bytes, and then prints the two versions, of the string, to the terminal.

fn main() {
let s: String = String::from("hello");
println!("String: {:?}", &s);
println!("Bytes: {:?}", &s.as_bytes());
}
String: "hello"
Bytes: [104, 101, 108, 108, 111]

Wasm “Hello World!” example

Given all of this information, how would we write a “Hello World!” application in Wasm, for the Web? For example, how would we pass strings back and forth between the user’s interface and the Wasm execution environment?

Image for post
Image for post
Wrestling pictorial that looks surprisingly like a Crustacean. Coincidence? I think not.

Bindgen

Wasm-bindgen is a build-time dependancy for Rust. It is able to generate Rust and JavaScript code at compile time. It can also be used as an executable, called bindgen in the command line. Essentially, the wasm-bindgen tool allows JavaScript and Wasm to communicate high-level JavaScript objects like strings. As opposed to exclusively communicating number data types (Rustwasm.github.io, 2019).

Memory

Temporary JS objects on the “stack”

Short-term JavaScript objects are pushed on to the stack, and their indices (position in the stack, and length) are passed to Wasm. A stack pointer is maintained to figure out where the next item is pushed (GitHub — RustWasm , 2020).

JsValue

The Rust codebase of the wasm-bindgen library, itself, uses a special JsValue. A hand-written exported function, like the one pictured below, can take a reference to this special JsValue.

#[wasm_bindgen]
pub fn foo(a: &JsValue) {
// ...
}

wasm-bindgen generated Rust

The Rust code that #[wasm_bindgen] generates, in relation to the hand-written Rust above, looks something like this.

#[export_name = "foo"] 
pub extern "C" fn __wasm_bindgen_generated_foo(arg0: u32) {
let arg0 = unsafe {
ManuallyDrop::new(JsValue::__from_idx(arg0))
};
let arg0 = &*arg0;
foo(arg0);
}

wasm-bindgen generated JavaScript

// foo.js
import * as wasm from './foo_bg';
const heap = new Array(32).fill(undefined);
heap.push(undefined, null, true, false);
let stack_pointer = 32;
function addBorrowedObject(obj) {
stack_pointer -= 1;
heap[stack_pointer] = obj;
return stack_pointer;
}
export function foo(arg0) {
const idx0 = addBorrowedObject(arg0);
try {
wasm.foo(idx0);
} finally {
heap[stack_pointer++] = undefined;
}
}

The heap

As we can see the JavaScript file imports from the Wasm file.

self.global(&format!("const heap = new Array({}).fill(undefined);", INITIAL_HEAP_OFFSET));
Image for post
Image for post

Long-lived JS objects

Here we will talk about the second half of management of JavaScript objects, again referencing the official bindgen documentation (Rustwasm.github.io, 2019).

// foo.rs
#[wasm_bindgen]
pub fn foo(a: JsValue) {
// ...
}
import * as wasm from './foo_bg'; // imports from wasm fileconst heap = new Array(32);
heap.push(undefined, null, true, false);
let heap_next = 36;
function addHeapObject(obj) {
if (heap_next === heap.length)
heap.push(heap.length + 1);
const idx = heap_next;
heap_next = heap[idx];
heap[idx] = obj;
return idx;
}
const encoder = new TextEncoder();
const encoded = encoder.encode('Tim');
encoded
// Uint8Array(3) [84, 105, 109]

Concrete examples

As the bindgen documentation says. “With the addition of wasm-pack you can run the gamut from running Rust on the web locally, publishing it as part of a larger application, or even publishing Rust-compiled-to-WebAssembly on NPM!”

Wasm-pack

Image for post
Image for post
https://rustwasm.github.io/wasm-pack/

Wasm-pack (client side — web)

Here is an example of how wasm-pack facilitates string concatenation using Wasm on the web.

#System housekeeping
sudo apt-get update
sudo apt-get -y upgrade
sudo apt install build-essential
#Install apache
sudo apt-get -y install apache2
sudo chown -R $USER:$USER /var/www/html
sudo systemctl start apache2
#Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
#Install wasm-pack
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
cd ~
cargo new --lib greet
cd greet
[lib]
name = "greet_lib"
path = "src/lib.rs"
crate-type =["cdylib"]
[dependencies]
wasm-bindgen = "0.2.50"
use wasm_bindgen::prelude::*;#[wasm_bindgen]
extern {
fn alert(s: &str);
}
#[wasm_bindgen]
pub fn greet(name: &str) {
alert(&format!("Hello, {}!", name));
}
wasm-pack build --target web
<html><head>
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous" />
<script type="module">import init, { greet } from './greet_lib.js';async function run() {await init();var buttonOne = document.getElementById('buttonOne');buttonOne.addEventListener('click', function() {var input = $("#nameInput").val();alert(greet(input));}, false);}run();</script>
</head>
<body>
<div class="row">
<div class="col-sm-4"></div>
<div class="col-sm-4"><b>Wasm - Say hello</b></div>
<div class="col-sm-4"></div>
</div>
<hr />
<div class="row">
<div class="col-sm-2"></div>
<div class="col-sm-4">What is your name?</div>
<div class="col-sm-4"> Click the button</div>
<div class="col-sm-2"></div>
</div>
<div class="row">
<div class="col-sm-2"></div>
<div class="col-sm-4">
<input type="text" id="nameInput" placeholder="1" , value="1">
</div>
<div class="col-sm-4">
<button class="bg-light" id="buttonOne">Say hello</button>
</div>
<div class="col-sm-2"></div>
</div>
</body>
<scriptsrc="https://code.jquery.com/jquery-3.4.1.js" integrity="sha256-WpOohJOqMqqyKL9FccASB9O0KwACQJpFTUBLTYOVvVU=" crossorigin="anonymous">
</script>
</html>
cp -rp pkg/* /var/www/html/
Image for post
Image for post
Strings in WebAssembly
Strings in WebAssembly

Wasm-pack (server side — Node.js)

Now that we have seen this in action using HTML/JS and Apache2, let’s go ahead and create another demonstration. This time in the context of Node.js, following wasm-pack’s npm-browser-packages documentation.

sudo apt-get update
sudo apt-get -y upgrade
sudo apt-get -y install build-essential
sudo apt-get -y install curl
#Install Node and NPM
curl -sL https://deb.nodesource.com/setup_13.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install npm
#Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
#Install wasm-pack
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf |
sudo apt-get install pkg-config
sudo apt-get install libssl-dev
cargo install cargo-generate
cargo generate --git https://github.com/rustwasm/wasm-pack-template
mod utils;
use wasm_bindgen::prelude::*;
// When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
// allocator.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;
#[wasm_bindgen]
extern {
fn alert(s: &str);
}
#[wasm_bindgen]
pub fn greet() {
alert("Hello, tpmccallum-greet!");
}
wasm-pack build --scope tpmccallum
wasm-pack login
cd pkg
npm publish --access=public
cd ~
npm init wasm-app create-wasm-app
npm i @tpmccallum/tpmccallum-greet
import * as wasm from "tpmccallum-greet";

wasm.greet();
npm install
npm start
Image for post
Image for post

Wider applications for Wasm

It is anticipated that “WebAssembly will find a wide range of uses in other domains. In fact, multiple other embeddings are already being developed: for sandboxing in content delivery networks, for smart contracts or decentralised cloud computing on blockchains, as code formats for mobile devices, and even as mere stand-alone engines for providing portable language runtimes” (Rossberg et al., 2018).

Shoutout to WebAssembly Summit

On the February 10, 2020 a WebAssembly Summit will be held in Mountain View CA.

References

Blandy, J. and Orendorff, J. (2017). Programming Rust. O’Reilly Media Inc.

Author

Timothy McCallum

Wasm

Dedicated to curating the highest quality WebAssembly…

Timothy McCallum

Written by

Researcher and Open Source Core Developer at secondstate.io — Book contributions Mastering Ethereum, Building Blockchain Apps — Mentor Google Summer of Code…

Wasm

Wasm

Dedicated to curating the highest quality WebAssembly (Wasm) information, in an unofficial capacity. Encouraging writers, developers and researchers to share everything from innovative business ideas & Wasm use cases, right through to technical insights, documentation & code.

Timothy McCallum

Written by

Researcher and Open Source Core Developer at secondstate.io — Book contributions Mastering Ethereum, Building Blockchain Apps — Mentor Google Summer of Code…

Wasm

Wasm

Dedicated to curating the highest quality WebAssembly (Wasm) information, in an unofficial capacity. Encouraging writers, developers and researchers to share everything from innovative business ideas & Wasm use cases, right through to technical insights, documentation & code.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store