Introducing The Michelson Execution Engine

Background

I joined the Tocqueville Group (“TQ”) as a software developer back in February with the intent to gain concrete blockchain experience and do the sort of deep and comprehensive learning I would never be able to do once I start full time at Google in September. With a nascent blockchain like Tezos, which just launched its mainnet in September of last year, there is a world of projects that are waiting to be built.

I settled upon creating a Michelson test framework, after discovering that the tools for Michelson could be rather hard to find or non-existent at times. I wanted to create a way to easily test smart contracts and learn more about Michelson as a language. In the process, I spoke to incredibly talented engineers, learned to appreciate OCaml and functional programming, and created an execution engine that I’m proud to introduce today. The sections below discuss the details of the project.

From Test Framework To Execution Engine

You mentioned that you set out to create a test framework, so how did it become an execution engine? What’s the difference?

My initial desire to create a test framework arose from the observation that testing Michelson contracts is a bit cumbersome. You can execute contracts using a remote procedure call (RPC) on the command line, and I initially wrote a script to do that and test the command-line outputs, but there are many limitations to that. For one, it’s incredibly slow, and you can only test the output of your execution — there isn’t a way to observe or manipulate what goes on throughout the process. Clearly, I needed something faster and more versatile.

Both of these problems are solved by parsing and interpreting Michelson files directly instead of navigating through the RPC. Luckily, the implementation of the interpreter can be found in the Tezos source code. I took the parser and interpreter and packaged it neatly with far fewer dependencies (much of the protocol material wasn’t needed) and, at the end, sprinkled on some helpful factory and printing methods.

That whole process took months of my time, and afterwards, I realized that the heavy lifting was actually all done, and what I had in front of me was actually the test framework I needed all along.

Let me explain. I could have created a test framework for Michelson like JUnit or Quickcheck that aided in creating unit tests or property tests for Michelson files, but OCaml already has some built-in libraries for this, such as OUnit. If I wanted to use OUnit, all I needed was for a way to easily transform OCaml types into Michelson types, and back. That way, I could parse the program, do some initial sanity tests, execute the program, and convert the resulting data back into OCaml types to write tests for them. This is, in essence, what the Execution Engine does.

Project Scope and Details

The Michelson Execution Engine can be found here. Documentation is on the wiki tab of the repository page.

Some Examples of execution and testing:

I tried to make the API as simple and intuitive as possible, so I hope that a lot of what the program does can be inferred from the names. This example can be found in the examples tab of the repository.

Environment, Context, Execution Context

The Environment is a module that contains the parameter and storage types of the contract. The Context data type contains blockchain-specific parameters (i.e, what contracts are initialized with what balance, what is the gas limit during execution). The Execution Context data type contains contract-specific parameters, i.e who the senders are, what the parameter and storage are, etc.

Fileparser, Engine, Data factory, Printer

These modules do a lot of the heavy lifting.

The Fileparser module reads in a Michelson File and returns a representation of the program as a set of instructions called a lambda.

The Engine module can interpret that lambda, or it can also be used to parse and interpret the program directly, without needing the Fileparser. However, using the stepping functionality (executing one instruction at a time) requires users to parse the program using the Fileparser first.

The Data Factory module is used to create the data types used in the contract, which can sometimes be complicated, as with maps and key hashes.

The Printer Module prints out some of the more intricate data structures in a readable format. The primary use-case is printing out the remaining instructions and the contents of the stack.

Use Cases and Further Work

The primary use case that I imagined for this execution engine is as a subcomponent of a test framework. For instance, in the example above, I use my engine to execute the smart contract’s main method with initial parameters and storage, and then I use the OUnit test framework to check that there are no operations returned from executing this smart contract.

There is an incredible amount of work left to be done. Testing of the framework is the priority, and I intend to create interface files for all of the files in the repository. A list of the work left to be done can be found on the README of the repo.

Ultimately, I believe this project builds upon an important goal of the Tezos codebase, which is to separate and modularize the individual components of the codebase. I hope that this project can help to expedite that process.

Contributions are very welcome!