Hackathon Challenge: Build a Compiler for A Structured Data Definition Language for Solidity

Loopring is sponsoring CryptoBazar’s upcoming September hackathon, and here is our Hackathon challenge: design a solidity struct definition language and build a compiler for it to help solidity-based smart contract to serialize and deserialize solidity structs.

The Challenge

Solidity is a language for writing smart contracts on Ethereum platform. It offers a set of built-in types. When writing smart contract, people usually define their public/external API like this (example from Loopring Protocol 1.0):

function submitRing(
  address[2][] addressList,
  uint[7][] uintArgsList,
  uint8[2][] uint8ArgsList,
  bool[] buyNoMoreThanAmountBList,
  uint8[] vList,
  bytes32[] rList,
  bytes32[] sList,
  address ringminer,
  address feeRecepient
) public;

The issues are:

1. Many of these parameters are optional, engineers at Loopring usually has to give these optional parameters their default values in a submitRing transaction. Therefore the optional parameters’ default values do consume gas, we want to avoid it.
2. There is a limit on the number of parameters each function can have. Therefore, in Loopring 1.0, we have to organize parameters of the same type into an array and give it a terrible name, such as uint8ArgsList. We want to name each parameter with its own desired name to make the code more readable.
3. There is no backward compatibility if we change the method’s parameters in a newer version.

Expected Solution

We learn from Google’s Protocol Buffer (protobuf) and Facebook’s thrift, and imagine we can define our domain models in a protobuf/thrift like language and write a compiler to generate a set of solidity files that contain our domain models in solidity struct as well as a serialization function to serialize the struct into a byte array (bytes) and a deserialization function to deserialize a bytes into the domain struct.

Let’s take the following struct in solidity as an example:

struct SimpleOrder {
  address   owner;
  address   tokenS;
  address   tokenB;
  uint      amountS;
  uint      amountB;
  uint      validSince;
  uint tokenSpendableS;
  uint tokenSpendableFee;
}
function submitSimpleOrder (
  address   owner,
  address   tokenS,
  address   tokenB,
  uint      amountS,
  uint      amountB,
  uint      validSince,
  uint tokenSpendableS,
  uint tokenSpendableFee) {
  SimpleOrder order = new SimpleOrder(
    owner,
    tokenS,
    tokenB,
    amountS,
    amountB,
    validSince,
    tokenSpendableS,
   tokenSpendableFee);
  // ...
}

We expect that all fields are optional. We can define such a struct in a prosol language yet to be invented:

// simple_order.prosol
structure SimpleOrder {
  address   owner = 1;
  address   tokenS = 2;
  address   tokenB = 3;
  uint      amountS = 4;
  uint      amountB = 5;
  uint      validSince = 6;
  uint tokenSpendableS = 7;
  uint tokenSpendableFee = 8;
}

The numbers behind field names are the fields’ indices which uniquely identify the fields in the serialized byte array, not their names.

A compiler can be made available to compile simple_order.prosol into a file called generated/SimpleOrderLib.sol with the following content:

pragma solidity 0.4.24;
pragma experimental "v0.5.0";
pragma experimental "ABIEncoderV2";
import "path/to/Prosol.sol"; // this has some basic methods.
/// Automatically generated from simple_order.prosol
/// Do not change manually.
library SimpleOrderLib {
  struct SimpleOrder {
    address   owner;  // pos=1;
    address   tokenS;  // pos=2;
    address   tokenB;  // pos=3;
    uint    amountS;  // pos=4;
    uint    amountB;  // pos=5;
    uint    validSince;  // pos=6;
    uint tokenSpendableS;  // pos=7;
    uint tokenSpendableFee;  // pos=8;
  }
function toBytes(SimpleOrder simpleOrder)
    pure
    returns (bytes output) {
    // generated code below
  }
function toSimpleOrder(bytes input)
    pure
    returns (SimpleOrder simpleOrder) {
    // generated code below
  }
}

With this generated library, we can change the submitSimpleOrder function to something like this:

pragma solidity 0.4.24;
import "generated/SimpleOrderLib.sol";
using SimpleOrderLib for bytes;
using SimpleOrderLib for SimpleOrder;
function submitSimpleOrder(bytes input) {
  SimpleOrderLib.SimpleOrder order = input.toSimpleOrder();
  bytes output = order.toBytes();
  // ...
}

Default Values

When serializing a struct into a bytes, all fields with default values should be omitted to save space.

Support Embedded Structure

The compiler also needs to support embedded structures. Substructures should have their own indexing space. For example:

// simple_order.prosol
structure Spendables {
  uint tokenSpendableS = 1;
  uint tokenSpendableFee = 1;
}
structure SimpleOrder {
  address   owner = 1;
  address   tokenS = 2;
  address   tokenB = 3;
  uint      amountS = 4;
  uint      amountB = 5;
  uint      validSince = 6;
  Spendables spendables = 7;
}

We may need to restrict the number of fields per structure in order to make sure the indices themselves do not take too many bits. After all, reducing the size of transaction data payload and gas consumption is our objective.

Support Repeated Fields

Repeated fields should be compiled into an array. In the example below, the spendables fields should have a solidity type of Spendables[].

// simple_order.prosol
structure SimpleOrder {
  address   owner = 1;
  address   tokenS = 2;
  address   tokenB = 3;
  uint      amountS = 4;
  uint      amountB = 5;
  uint      validSince = 6;
  repeated Spendables spendables = 7;
}

To stay up-to-date with Loopring, please sign up for Loopring’s Bi-Weekly Update, and find us here:

• Twitter: twitter.com/loopringorg
• Reddit: reddit.com/r/loopringorg
• Telegram: t.me/loopring_en
• Telegram: t.me/loopringfans (Chinese)
• Discord: discord.gg/KkYccYp
• StackOverflow: stackoverflow.com/c/loopring
• Kakao: open.kakao.com/o/gJbSZdF (Korean)