The future of Web Assembly (WASM): the hardware-execution revolution!
This article will take you on a deep technical dive inside WebAssembly (Wasm).
We will be compiling a simple Rust program to Wasm, and then deploying it on a stand-alone Wasm Virtual Machine (called WAVM). We will also be writing a C program, and then compiling that to Wasm and deploying it on x86_64 hardware (macOS Catalina). The latter task will be performed using Fastly’s native WebAssembly compiler and runtime called Lucet.
Please note: This article is more than just a technical dive into Wasm. This article also discusses the future opportunities of Wasm, in a much broader context.
We will be discussing the use of Wasm as part of blockchain (global decentralised computing) implementations, hardware implementations (portable binaries) and service-oriented architecture (SOA) implementations.
By the end of this article you will have a good understanding of Wasm fundamentals. In addition, you will have learned how to access free resources which will help you to write, compile and execute your own hand written Wasm code. Let’s get started.
What is Wasm?
WASM is a machine-close, platform-independent, low-level, assembly-like language (Reiser and Bläser, 2017). Wasm addresses the problem of safe, fast, portable low-level code on the web (Rossberg et al., 2018).
The Wasm computational model is based on a stack machine, in that instructions manipulate values on an implicit operand stack, consuming (popping) argument values and producing or returning (pushing) result values (webassembly.github.io, 2019).
Wasm’s growth
The following graph shows the number of “WebAssembly” scholarly publications during the last few years.
As you can see there is a steep rise in academic “WebAssembly” papers and there has also been a steady rise in the number of papers which contain, both, the words “WebAssembly” and “Blockchain”. We will discuss Wasm in the blockchain a little later on. Right now let’s discuss in-browser Wasm implementations.
In-browser Wasm implementations
WASM’s design enables evolutionary web development (Webassembly.org, 2019). There are many impressive in-browser implementations of Wasm.
One example is this online Wasm maze game.
When compiled, this in-browser game, is less than 2048 bytes in size!
Another example, of an excellent in-browser Wasm implementation, is this compression/decompression software called wasm-flate.
wasm-flate is currently the fastest compression and decompression software in the browser. This kind of in-browser Wasm execution provides opportunities for web developers to seamlessly integrate powerful new features into their web applications. Wasm developments like these, mean that end users are no longer required to install, or jump between, third-party system-level applications.
Could in-browser Wasm applications like wasm-flate ultimately displace competing traditional system-level applications like WinZip?
Wasm in blockchain implementations
Bitcoin and Ethereum (and some other blockchain implementations) use a stack-based architecture which has similarities to WebAssembly’s stack-based architecture.
There are of course some differences in each of the unique stack-based virtual machines. For example equivalents to the well known stack-item-duplication operations such as Bitcoin’s OP_DUP opcode and Ethereum’s DUP1 to DUP16 opcodes are not found in Wasm.
Pictured: Ethereum Yellow Paper’s duplication operations.
Thankfully, Wasm offers a fixed amount of local variables for each Wasm function. These variables store information inside single index spaces which are local to that specific function. Point being, there are other ways of emulating certain stack behaviours.
Another important difference relates to the number of items which can be pushed onto the stack, per operation. If you look closely at the Ethereum Yellow Paper (pictured above) you will notice two columns labelled δ and α.
The column labelled δ represents the number of items to be removed from the stack. The next column over, labelled α, represents the number of additional items to be placed on the stack. As you can see each of the operations on the Ethereum Virtual Machine (EVM) is able to push many items onto the stack. In the above example DUP16 is capable of pushing 17 items onto the stack.
In the current version of Wasm however, only one single result value can be pushed on to the stack by a single instruction*see Update 20191128 in the Appendix* (webassembly.github.io, 2019).
With many other nuances like this, it goes without saying that building a compiler which is capable of converting any high level blockchain smart contract source code to executable Wasm-flavoured code is a seriously complex and onerous task.
Developers from SecondState recently built a compiler called Soll (video demonstration available here) which was the first compiler to allow Ethereum Solidity smart contracts to be, compiled, deployed and interacted with on the Ethereum flavoured Wasm (Ewasm) testnet.
Trail-blazing work, such as this, marks the beginning of exchanging digital value and data, as well as, engaging in rule-based interactions between devices across the decentralised web. The weaving of browser-based devices into blockchain’s already decentralised architecture could make permission-less, censorship-resistant, borderless, yet secure, web-based, transacting mainstream.
An article written after SecondState’s presentation at Devcon5 (an Ethereum conference for developers) illustrated that SecondState are also considering building a compiler from Ethereum’s intermediate language “Yul” to llvm to Ewasm.
This additional new work potentially allows for a wider variety of smart contracts (written in languages such as C++, Rust, Vyper etc.) to be deployed to Ethereum’s Wasm blockchain implementation.
It quickly becomes apparent that introducing new languages (across separate sections of compiler toolchains) leads to enormous potential in terms of language agnostic collaboration.
This is one of the great future benefits, which Wasm brings to the table.
Wasm — closer to the hardware
Did you know that Wasm is more than just bytecode for web browsers or blockchain VMs?
The web spans not only many web browsers, but different device classes, machine architectures and operating systems. Code, which targets the web, must be hardware and platform independent to allow applications to run across all hardware types with the same behaviour (Rossberg et al., 2018).
Wasm execution environments can include minimal shells, mobiles, desktops and IoT devices. Wasm can potentially drive everything from microchips to full-blown data-centres (Webassembly.org, 2019).
Solomon Hykes, who co-founded Docker said earlier this year that “if Wasm+WASI existed in 2008, we wouldn’t have needed to created Docker. That’s how important it is. Webassembly on the server is the future of computing.”
It is logical that Wasm will appeal to the largest percentage of the global software development community. Wasm’s design brings about incredible speed, flexibility and portability, and as such, it may be easy for one to surmise that Wasm will play some part in the majority of each of the world’s upcoming computing solutions (both on and off the Web).
Programming Wasm
Let’s dive right in and cut some code. We are about to write a Rust program, compile it, and then deploy it on a stand-alone Wasm VM.
Installing Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shsource $HOME/.cargo/env
Adding components
rustup component add rls rust-analysis rust-srcCreate a new Rust project
cd ~
cargo new --lib add_numbers_via_wavm
cd add_numbers_via_wavmEdit the Cargo.toml file; add the lib section to the end of the file, as shown below
[lib]
name = "adding_lib"
path = "src/adding.rs"
crate-type =["cdylib"]A quick note about “cdylib”
“If you’re producing a library that you intend to be used from C (or another language through a C FFI), there’s no need for Rust to include Rust-specific stuff in the final object code. For libraries like that, you’ll want to use the cdylib crate type in your Cargo.toml”(Doc.rust-lang.org, 2019)
We now add the necessary Wasm software and configuration
rustup target add wasm32-wasi
rustup override set nightlyNow, create a new file called ~/.cargo/config and place the following build text into this newly created config file
[build]
target = "wasm32-wasi"Please note: rust-lang’s `wasm32-unknown-wasi` was recently renamed to just `wasm32-wasi`
Rust source code
Create a file called adding.rs in the /home/ubuntu/add_numbers_via_wavm/src directory and fill it with the following Rust code
#[no_mangle]
pub extern fn main(a: i32, b: i32) {
let z = a + b;
println!("The value of x is: {}", z);
}Compiling source code
This will create an adding_lib.wasm file in the /home/ubuntu/add_numbers_via_wavm/target/wasm32-wasi/release directory
cargo build --releaseWe will be executing this wasm file in a minute. First we have to install our WebAssembly Virtual Machine.
Creating the stand-alone Wasm Virtual Machine
We already know that software like wasm-pack allows you to integrate your Rust- generated WebAssembly with JavaScript and that software such as wasm-bindgen facilitates high-level interactions between wasm modules and JavaScript. But what we are about to do here is completely different.
We are not using Javascript, or Node.js or running anything in a browser. We are executing our Rust program inside a stand-alone WebAssembly Virtual Machine which has been specifically designed for use in non-web applications.
Installing the Virtual Machine called WAVM
sudo apt-get install gcc
sudo apt-get install clang
wget https://github.com/WAVM/WAVM/releases/download/nightly%2F2019-11-04/wavm-0.0.0-prerelease-linux.deb
sudo apt install ./wavm-0.0.0-prerelease-linux.debExecuting Wasm on the VM
Let’s try and execute the (Rust to Wasm) code, which we just compiled, using the following WAVM command
wavm run --abi=wasi --function=main ~/add_numbers_via_wavm/target/wasm32-wasi/release/adding_lib.wasm 2 2The value of x is: 4
As you can see above, we are able to pass in two values (2 and 2) and the WebAssembly Virtual Machine (WAVM) is able to calculate the sum total and return the answer to us via the console.
This is incredible exciting!!!
Deploying a C program on x86_64 hardware.
We have just executed Rust/Wasm inside a stand-alone VM. Let’s now deploy a C program on x86_64 hardware (macOS Catalina) using Lucet. Lucet is not a VM, but a a WebAssembly compiler and runtime which allows Wasm to execute on the server side.
Like WAVM, Lucet also supports the WebAssembly System Interface (WASI) — a new proposed standard for safely exposing low-level interfaces to the filesystem, networking, and other system facilities
We commence this demonstration by compiling Lucet from source.
We then create the hello world C source code file (as pictured below) …
#include <stdio.h>
int main(int argc, char* argv[])
{
if (argc > 1) {
printf("Hello from Lucet, %s!\n", argv[1]);
} else {
puts("Hello, world!");
}
return 0;
}… change to location where Lucet was installed …
cd /opt/lucet/bin… compile the C code to Wasm …
./wasm32-wasi-clang ~/lucet/tpmccallum/hello.c -o ~/lucet/tpmccallum/hello.wasm… and see output along these lines.
We then pass the hello.wasm file into the next command, which produces hello.so
lucetc-wasi ~/lucet/tpmccallum/hello.wasm -o ~/lucet/tpmccallum/hello.soThe output from the previous command looks like this.
To complete this demonstration, we finally run the following command which executes the program as a Mach-O 64-bit dynamically linked shared library x86_64 file (on macOS Catalina)
What we have essentially done here is executed server-side Wasm.
Other Wasm compilers and runtimes
As mentioned previously, there are a growing number of Wasm compilers and runtimes. These include, Intel’s Wasm Micro runtime and the Wasm Virtual Machine. In addition to these projects, the WASMER project allows you to compile everything to WebAssembly, then run it on any OS or embed it into other language (Wasmer.io, 2019). For example you could write code in Go, Rust, Python, Ruby, PHP, C, C++ and C#. Then compile that code to Wasm, and then embed that Wasm code back inside any of the aforementioned languages. WASMER is also working on creating binaries that will execute on any platform.
The inner workings of Wasm
Let’s take a brief look at the inner workings of Wasm and then finish up by providing some resources for you to create your first hand-written Wasm application.
Numeric instructions are divided by value type. For each type, several subcategories can be distinguished:
- Unary Operations — consume one operand and produce one result of the respective type
- Binary Operations — consume two operands and produce one result of the respective type
- Comparisons — consume two operands of the respective type and produce a Boolean integer result
- Tests — consume one operand of the respective type and produce a Boolean integer result
- Conversions — consume a value of one type and produce a result of another
Memory is accessed using the load and store instructions. All values are read and written in little endian byte order.
Variable instructions, provide access to local and global variables.
Control instructions include if, loop and other instructions which have an effect over the control of code execution.
The global state
The store represents all global state that can be manipulated by WebAssembly programs.
The store, keeps an individual index position for every function instance, table instance, memory instance, and global instance which has been allocated during the life time of the abstract machine. Each of these separate index positions is referenced/accessed through the use of an address.
Addresses are dynamic, globally unique references to runtime objects (webassembly.github.io, 2019).
Value types classify the individual values that WebAssembly code can compute with and the values that a variable accepts. The types i32 and i64 classify 32 and 64 bit integers, respectively. Integers are not inherently signed or unsigned, their interpretation is determined by individual operations (webassembly.github.io, 2019). The types f32 and f64 represent floating point values.
Each is explicitly encoded by a single byte, as shown below.
Online Wasm code editor
Now that you have a better understanding of the Wasm fundamentals, it is time to write and deploy your own hand-written Wasm code. We can use the online web-based WebAssemblyStudio application to perform this task.
Wasm has a text file format “.wat”, and a binary file format “.wasm”. The WebAssemblyStudio application (pictured above), allows us to create Wasm applications in a variety of source formats. Including Wasm’s aforementioned text format .wat.
Here is our own simple function, written in Wasm text, inside the editor.
So what does this function do?
- As you can see above, line one defines the module. A valid module can be as little as the text “(module)”
- The second line defines the function called “add”. This function takes two i32 arguments, “firstValue” and “secondValue”. It returns a single i32 variable
- The third line of this code pushes the value of firstValue on to the stack
- The fourth line pushes the value of secondValue on to the stack
- Line 5 pops the two current items from the stack and then calculates the sum of those two values and pushes that sum value onto the stack
- When a function explicitly declares a return value, in this case (result i32), the last remaining item on the stack is always earmarked as the value which the function has promised to return
You may be wondering how the needs of each individual operation’s arguments (and return commitments of the overall function) are accounted for, in terms of the perfect amount of items on the stack.
The relationship between operations and items is able to be calculated ahead of time thanks to “function types” which classify the signature of functions. More specifically, function types define how many items each individual operation “pops off” the stack and how many items the operation then “pushes onto” the stack. This information allows explicit code validation to be performed.
Below is an illustration of the add operation (which pops off two i32 values and pushes on one i32 value).
i32.add
------------------------
[pops off] [pushes]
[i32 i32] -> [i32]Converting WAT to Wasm by hand
Whilst online products like the WebAssemblyStudio application handle all of the compiling and transpiling for us. We can also perform tasks, like creating Wasm output, in the command line.
tpmccallum$ ./wat2wasm adding_numbers.wat -o adding_numbers.wasmYou may recall from our command line C to Wasm example above that the Wasm binary formatted outputs were illegible to the human eye. These executable binary files are not designed to be viewed natively by your operating system (i.e. vi or emacs).
Thankfully we can rely on pre-built Wasm software libraries to convert Wasm code.
Wabt pronounced “wabbit” is an excellent library of Wasm tools. Wabt will perform tasks including, but not limited to converting Wasm text to Wasm binary (wat2wasm), converting binary back to text (wasm2wat), counting opcode usage for instructions (wasm_opcodecnt), converting Wasm to C (wasm2c) and more.
Take the following command for example.
tpmccallum$ ./wat2wasm adding_numbers.wat -vSee appendix A.1 for the full output of this assembly-based code
Results
Returning to our hand-written Wasm demonstration. If we click the Build & Run button in the WebAssemblyStudio application, we see that the function has added “firstValue” and “secondValue” and that it now returns the sum of these values “2”.
Conclusion
It is still very early days for Wasm. Notwithstanding, many popular programming languages such as C, C++, Rust, Go and C# can already compile their source code into production ready Wasm code.
This unprecedented portability is huge, in terms of developer adoption and collaboration.
We know that there are a number of very impressive in-browser Wasm applications in circulation. But there is also a growing number of Wasm compilers and runtimes which allow Wasm to execute outside of the web browser; closer to the hardware.
It is inevitable that Wasm flavoured compilers will make their way closer and closer to the hardware. This is a breeding ground for producing highly-efficient, portable and accessible, discrete units of functionality.
Wasm has all of the makings of the next-generation’s service-oriented architecture (SOA). It can target specific outcomes, can be self contained, provides abstraction and can easily share and consume other underlying units of functionality.
This is a very exciting area for collaboration. Projects are under very heavy development and we are destined to see great things come from all of this hard work.
References
Doc.rust-lang.org. (2019). cdylib crates for C interoperability — The Edition Guide. [online] Available at: https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/cdylib-crates-for-c-interoperability.html [Accessed 6 Nov. 2019].
GitHub/appcypher. (2019). Awesome WebAssembly Languages. [online] Available at: https://github.com/appcypher/awesome-wasm-langs [Accessed 27 Oct. 2019].
Reiser, M. and Bläser, L., 2017, October. Accelerate JavaScript applications by cross-compiling to WebAssembly. In Proceedings of the 9th ACM SIGPLAN International Workshop on Virtual Machines and Intermediate Languages (pp. 10–17). ACM.
Rossberg, A., Titzer, B., Haas, A., Schuff, D., Gohman, D., Wagner, L., Zakai, A., Bastien, J. and Holman, M. (2018). Bringing the web up to speed with WebAssembly. Communications of the ACM, 61(12), pp.107–115.
Wasmer.io. (2019). Wasmer — The Universal WebAssembly Runtime. [online] Available at: https://wasmer.io/ [Accessed 4 Nov. 2019].
webassembly.github.io. (2019). WebAssembly Specification. [online] Available at: http://webassembly.github.io/spec/core/ [Accessed 23 Oct. 2019].
Webassembly.org. (2019). Non-Web Embeddings — WebAssembly. [online] Available at: https://webassembly.org/docs/non-web/ [Accessed 28 Oct. 2019].
Appendix
Update 20191128
Wasm instruction sequences can now consume and produce an arbitrary number of stack values. Not just push one resulting stack value, as was previously the case. The Multi-Value proposal who’s ongoing implementation is explained well here is currently at phase 3 of Wasm standardization process.
Appendix A.1
tpmccallum$ ./wat2wasm adding_numbers.wat -v
0000000: 0061 736d ; WASM_BINARY_MAGIC
0000004: 0100 0000 ; WASM_BINARY_VERSION
; section "Type" (1)
0000008: 01 ; section code
0000009: 00 ; section size (guess)
000000a: 01 ; num types
; type 0
000000b: 60 ; func
000000c: 03 ; num params
000000d: 7f ; i32
000000e: 7f ; i32
000000f: 7f ; i32
0000010: 01 ; num results
0000011: 7f ; i32
0000009: 08 ; FIXUP section size
; section "Function" (3)
0000012: 03 ; section code
0000013: 00 ; section size (guess)
0000014: 01 ; num functions
0000015: 00 ; function 0 signature index
0000013: 02 ; FIXUP section size
; section "Export" (7)
0000016: 07 ; section code
0000017: 00 ; section size (guess)
0000018: 01 ; num exports
0000019: 03 ; string length
000001a: 6164 64 add ; export name
000001d: 00 ; export kind
000001e: 00 ; export func index
0000017: 07 ; FIXUP section size
; section "Code" (10)
000001f: 0a ; section code
0000020: 00 ; section size (guess)
0000021: 01 ; num functions
; function body 0
0000022: 00 ; func body size (guess)
0000023: 00 ; local decl count
0000024: 20 ; local.get
0000025: 00 ; local index
0000026: 20 ; local.get
0000027: 01 ; local index
0000028: 6a ; i32.add
0000029: 0b ; end
0000022: 07 ; FIXUP func body size
0000020: 09 ; FIXUP section size
