Optimizing the payloads of Omni Layer transactions
The Omni Layer is a platform to create, transfer and trade digital tokens. In total a value of nearly $ 2.5 billion is represented on the layer.
To do so, the protocol uses Bitcoin transactions to embed commands into the Bitcoin blockchain, which are interpreted by Omni aware software. For example, these commands could be “Create 1000 GoldTokens” or “Transfer 7 GoldTokens to Alice”.
Due to the rising popularity of the underlying layer, Bitcoin, fees for transactions have grown significantly. This post shows three steps we are going to take to reduce the size of Omni Layer payloads by up to 50 % or more.
A brief history of embedding data into the chain
Omni Layer transactions have the following properties:
- A transaction marker to identify an Omni Layer transaction
- A payload, which is interpreted by Omni aware software
- Depending on the transaction type, a destination to specify where tokens should be sent to
Over the years different on-chain data and payload embedding techniques were introduced. Back in 2013 data was encoded as Bitcoin addresses and there was a transaction output to 1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P, serving as marker or identifier for Omni Layer transactions. However, this was only intended as first step, as it essentially burned Bitcoin, which were not retrievable from these outputs.
Shortly after, there was still a marker output, but the actual payload data was encoded as public key and placed into m-of-n multi-signature scripts, where one of the other keys was a public key of the sender. This allows to consolidate and retrieve coins at some point later.
At the beginning of 2016 we activated a new technique on main net, which has become the industry standard since then. The transaction marker became part of the payload and payloads are embedded with OP_RETURN scripts. This has several benefits, such as smaller overall transaction size.
A typical Omni Layer transaction
Let’s look at a typical Omni Layer payload for a “Simple Send” transaction. In this transaction 5.00000000 Omni are transferred to a recipient and the following hex-encoded payload is used to do so:
The transaction marker is used to identify the transaction as Omni Layer transaction, the version and type fields specify the transaction type — in this case a transfer transaction. The token identifier specifies the tokens to send, and the amount specifies the number of tokens nominated in base units that are going to be transferred.
Step 1: shorter transaction markers
The transaction marker serves as identifier for Omni Layer transactions and while it’s very expressive to use a long marker, it isn’t necessary at all. In the first optimization step the marker size is cut in half and the new marker 0x6f6c is used, which stands for “ol”. This reduces the payload size to 18 byte:
Step 2: merging transaction fields
The version and type fields specify the transaction command to execute. However, a total of 4 byte could be used for 4294967296 different commands, which is a lot more than we ever need.
In the second optimization step the transaction fields version and type are merged, saving another 2 byte:
Step 3: using variable-length encoding
Transaction fields currently use a fixed length: the type field has a length of 2 byte, the token identifier uses 4 byte, and the amount field uses 8 byte, even if only a few bits are actually used.
To further optimize the payload, numbers are compressed by using the variable-length code LEB128. This allows to store an arbitrarily large integer in a small number of bytes. To do so, the number is first represented in binary and zero extended up to a multiple of 7 bits. The data is then split into groups of 7 bits length and stored as byte, where the remaining one bit signals whether there are more packets to come. This signaling bit is the essential element, as it allows to use a variable length instead of a fixed length.
In the example the payload size is reduced by another 7 byte to 9 byte:
Conclusion
It was shown that the size of an Omni Layer payload was reduced from 20 byte to 9 byte, which is less than 50 % of it’s original size.
The size savings depend on the numbers used in the payloads, but savings of 40–50 % on average are expected.
The code for the new payload format is currently under review and going to be merged into Omni Core, the reference client for the Omni Layer.
