WASM and VMs, specialized for blockchains

Published in

hdac_rizon

11 min readMar 7, 2020

Many blockchain platform projects adopt a programing language to draw contracts and a concept of the sandbox virtual machine (VM) to implement those projects for usability and scalability when applying the definition of smart contracts.

For instance, Ethereum has the Ethereum Virtual Machine (EVM) along with Solidity. Some may have a misguided belief that the EVM and Solidity must have each other, but EVM’s entry consists of byte codes which are not directly related to Solidity. It can use any language if the EVM can create byte codes of the same specs. Vyper is one of the languages that can create EVM byte codes.

Ethereum announced a model change and showcased EWASM, a WASM-based VM, to replace the EVM while planning 2.0. Nowadays. other platform projects are also jumping on the bandwagon with WASM.

This post will share thoughts on WASM-based VMs said to be the next model for blockchains, progresses made in those attempts and unique problems arising in the process. If you need more general knowledge on computer operation such as languages, VMs and byte codes or descriptions in easy and simple terms, those are just one search away.

Use of WebAssembly

WASM stands for WebAssembly [1]. As you can guess from the name, it is a standard code spec whose goal is to create performance similar to native assembly in the web environment. Assembly is the final level language which humans can read and write, the level right before the CPU language — OPCODEs. In other words, it aims to define the minimum intermediate stage to connect two different layers — upper layer web and core layer native which seem to be heterogenic.

In fact, OPCODEs that consist of hexadecimal numbers can be used for this. A short OPCODE is written for a range of low-level approaches such as in-line hooking, stack control and register control.

Is a binary that compiles WASM composed of OPCODEs? It is not. It is also composed of intermediate representations (IR) appropriate for conversion, though not specific CPU architecture operation specs. That is why various introductory documents use the expression “more native” while specs define it as Virtual Instruction-Set-Architecture (ISA) [2].

Therefore, WASM needs a VM to compile IRs into your CPU architecture or to analyze and execute like an interpreter. WASM and Java-JVM are similar in their basic functions, but starkly different in other features. WASM was designed to partially enable parallel compilation and execute IRs while simultaneously receiving streamed IRs in the network environment. It also has other design principles to accommodate web computing. One of them is the sandbox environment, which prevents security threats by restricting access that can affect the executed host environment such as file I/O, memory I/O and others.

Then why is WASM used on blockchain platforms which are different from the web? Let us look at simple use cases to understand what characteristics of WASM make it suitable for blockchains.

Cases to implement high performance functions on the web

Google Earth

When complicated 3D rendering is needed on the web, additional programs are usually set up. But WASM supports 3D rendering without adding new programs on the web by using its hardware graphic control — OpenGL. Google Earth is a key example of this.

Cases to transfer native programs to web programs

QT for WebAssembly

QT is a C++ Cross platform UI Framework used on various operation systems to assist development of GUI programs with one code. Recently, QT advanced its WASM framework further to support GUI web programs. Its efforts are still revolving around the experimental realm such as portability, but it shows possibilities of using WASM for high-performance GUI programs on the web.

Malware

There is malware that tries to mine cryptocurrencies by secretly using others’ computers. It advanced to the level of attempting mining by just accessing a webpage without setting up any program since it started to use WASM and Emscripten. It is not likely to cause serious problems such as file manipulation because WASM and VMs are designed for security, but it will surely make various attempts to circumvent security technologies.

Native language’s architecture and loading that solved platform dependency

In the perspective of CPU architecture, a native programing language produces a final program binary composed of OPCODEs which CPUs can directly analyze when it is compiled. OPCODEs can be executed only on designated target architecture while other architecture CPUs cannot execute the binary.

To resolve this problem, researchers are conducting active research on ways to add emulation layers to different types of CPU architecture on the OS. Windows 10 on ARM set up on a recently released PC limitedly supports x86 programs.

When creating a binary composed of WASM IRs with no architecture dependency, it can execute through VMs regardless of the architecture of an execution environment’s CPU. Standard specs dub this as ‘hardware-independent.’

In this process, VMs’ methods can be categorized into interpreting, JIT compiling and AOT compiling. Most VMs choose an optimal method for each situation. Further details are to be explored below in the part for optimization technology.

Compliers create the final program binary based on designated file formats for each OS as well as operation sets. Microsoft’s PE is the most well-known EXE and DLL, while Unix/Linux often use ELF and Mac uses Mach-O. OSs do not support file formats for each other, and users had to re-compile programs to use them on another OS.

WASM, on the other hand, is a standardized format that does not depend on platforms such as OSs, which is why it can be executed in any environment only if it is equipped with a VM. Standard specs refer to this characteristic as ‘platform-independent.’ Web is not an exception for this. It does not need browsers and can be executed along with an independent VM or integrated to other environments.

Of course, WASM VMs must be provided for each existing OS/CPU architecture.

Hardware-independence and platform-independence are general characteristics of programing languages designed based on VMs. But WASM opened a way to enhance flexibility for existing languages by designing more detailed IRs which can be broadly applied to various languages.

Most blockchain nodes still run on X86/AMD64 architecture generally used for server computing, which is why it would be illogical to put it up as an advantage of using a platform on blockchains. But there would be no issue if WASM contracts are executed in environments using ARM or a mobile OS as platforms advance.

VMs’ optimization method to enhance performance

Can WASM IR compliers and interpreters implemented based on standards have performance similar to a CPU-executed native binary? Popular engines such as Google v8 and Mozilla’s SpiderMonkey that implemented language standards including WASM and ECMA-262 have many additional factors other than language standards. Standards define design principles, grammar and execution results, not software model designs such as IR creation and execution processes. That is why quality VMs use various optimization methods to improve their performance.

Smaller, faster — Optimization of IR creation

Compliers usually have optimization logics creating codes to reduce their size or execute faster. There are cases where two different types of optimization are exclusive to each other. For example, quick execution often requires bigger code sizes, while smaller code sizes may slow down execution.

Parameters related to optimization vary — the number of general registers the target CPU architecture has, whether it supports the newest features such as hardware acceleration and what function call protocols it entails. The newest methods are explored in all areas from hardware to software.

Which optimization method should WASM compliers focus on to create an IR? It is not an easy decision, considering platform and hardware independence mentioned above. Optimization that reduces computing power consumption for execution in a web browsing environment could result in disadvantages in other environments. Fast execution does not guarantee a fast process as code transmission takes more time in poor network conditions.

Limitations of interpreting and highlight of runtime optimization — JIT & AOT compilation

Interpreters’ basic implementation level circulates an Abstract-Syntax-Tree (AST) or executes after analyzing IRs while sliding. This process is bound to be slow compared to native binaries that directly executes CPU OPCODEs.

Well-optimized interpreter engines are converted into CPU OPCODEs during the initial execution of an IR block. This is called Just-In-Time (JIT) compilation, and it is effective when applied to frequently executed parts. But it would become slower than simple interpreting when it is applied to a group of simple code units executed once or twice as compilation requires time and resources. Ahead-of-Time (AOT) compilation that converts the entire IRs before execution could be appropriate if there is enough time between reception and execution of a program.

WASM was designed taking account of the well-known optimization process described above.

It selects an optimization method for different targets and purposes. The design principle that obviates information of a target to eliminate dependency and the optimization selection method which requires information on the existing execution environment and the purpose cannot be fulfilled simultaneously. Many obstacles arise in automatically selecting the optimal case. In the end, WASM module developers need to consider various environments before selecting the best optimization method. They also need to develop proper optimization methods for WASM rather than applying the existing native compilation’s optimization methods to develop WASM VMs.

Is WASM the absolute solution for blockchains?

The fact that blockchains and WASM appeared and garnered attention around the same time may be a remarkable coincidence. Seeing compilers and VMs which used to appeal to few, now gaining limelight again brings me a great joy.

Blockchain platforms have execution environments quite different from existing environments. Trustless decentralized platforms require servers to execute codes written by an anonymous developer, but it can be highly risky considering the security of nodes. That is why security design principles including WASM’s sandbox environment are pivotal on decentralized platforms.

As WASM is considered for web computing, it would have been great if it incorporated browser engine experiences. Several issues must be addressed to apply them onto blockchain platforms.

Considerations for blockchain hinder development

Transaction costs

Some blockchain platforms impose charges on each IR to prevent spamming and Denial-of-Service attacks. Costs differ based on each operation, but they generally increase along with the number of operations to execute.

Optimization methods above bear larger code sizes for optimal execution speed. Smart contract executors would naturally want to reduce transaction costs, but node or service operators cannot ignore the execution speed.

Non-deterministic results

Deterministic results refer to a case that consistently shows the same output with the same entry. This part explores non-deterministic cases.

All distributed nodes must have the same execution result when a contract is executed with the same entry at the same time. This is a prerequisite to reach the final consensus, which is why smart contracts are not supposed to have non-deterministic outputs.

An easy example would be a case where a smart contract uses a function created by a random number that does not consider non-deterministic outputs. WASM’s standard definition also has non-deterministic cases [3]. The thread feature is one of them. This may appear when a specific memory is concurrently controlled through multi-threading.

At the end of the day, developers must be able to restrict and test deterministic cases for smart contract WASM modules. But it is not easy to do so. In some known cases, they adopt nonstandard methods such as limiting execution results by force or using non-standard compiler options to find a solution. Nevertheless, it is not easy for a receiver to guarantee that a creator of smart contract WASM modules observed the methods or that all non-deterministic threats disappeared.

Various methods such as static analysis and emulation on IR codes must be tested when a new contract is registered [4], and there is no other choice but to trust the result.

Various vulnerability threats hidden in the code written by a third party

As mentioned earlier, executing codes written by a third party on a server can be risky. If VMs have vulnerabilities, the execution result cannot be guaranteed or in an extreme case, the server control can be stolen.

In web computing, accessing malicious websites which attack vulnerabilities of a browser engine may launch a real attack on a client on the executing end. On blockchain platforms, it may threaten servers instead of clients. As if validating these concerns, another blockchain project found vulnerabilities in VMs [5]. Fortunately, their security research team found it and installed a patch before any actual damage occurred [6].

On top of the advantages listed above, WASM is emerging as a new entry route among vulnerability researchers [7][8]. But they must guard against threats as any increase in the number of new optimization features can be translated into more vulnerability threats.

Somewhere between conservative validation and liberal optimization

We have explored WASM standards and various optimization methods and explained why it is difficult to use implementation such as browsers’ WASM execution engines on blockchain platforms.

In the long run, the gap between development of WASM VMs on the web and that on blockchains is bound to widen. VMs and compilers cannot be completed in a short period with limited resources. High development costs must be taken into consideration.

Moreover, it would still be difficult to choose the direction of development even though there are enough resources and time. Should we take a conservative approach and opt for the maximum validation? Or should we be more liberal and continue to try new things for commercialization even if there are threats? Or is there any way to use a browser engine highly experienced in VM development?

Leaders have different strategies for their project as there is no right or wrong answer to this. Conservative leaders limit their project’s implementation to basic methods without adopting various optimization methods as they prioritize validation. But those who take risks and try optimization uses the llvm backend for IR optimization and even create machine codes at runtime by adopting JIT.

But no project has tried to directly use a browser engine. Would a browser engine developer developing a blockchain platform try it?

Conclusion

WASM VMs on blockchain platforms have many stumbling blocks. A lot of platforms pursue their own direction in technology development. Each has different purposes and strengths, which can lead to more creative technologies and thus, to gradual advancement of the industry. But some projects lack technical talents to pursue their own technology or may fall into the common trap whichever way they choose. This means that they do not have to cut off more than they can chew to develop their own technologies. For now, it is hard to predict the result.

Then how can we save development resources for blockchain’s unique elements, balance out intertwined interests such as competition between players and create a common interest so that everyone’s efforts do not go meaningless?

We will continue to share information on optimization methods while conducting meticulous validation. We should try not to go around in circles because of undefined fear but should also avoid taking unreasonable risks and harm stability of our projects. We are planning to conduct research on optimization and validation by looking into a tool that can analyze vulnerabilities of contracts or applying optimization that performs AOT compilation in advance on transactions of a block proceeding with consensus in an execution engine. Moreover, we would like to march into a more general direction so that everyone in the decentralized ecosystem can benefit from our HDAC platform.

We will continue to deliver technology-intensive development.