Transactionalization or How to design your CC (custom consensus) rules blockchain

Last year, I wrote about how to implement CC and starting from the faucet CC to the more complex CC examples, dozens of new CC have been made. It is likely that I have made more new blockchain consensus systems in the last year, than anybody else. maybe even more than everybody else combined.

However, there is no reason this needs to remain something only possible by a small number of people. This document assumes you are familiar with utxo based blockchains, most specifically how vouts in one tx become vins in the tx that spends it. Other than that, it assumes that you are able to write code, otherwise how can you design something that can be coded?!

As with anything, anybody can make a complex design to solve a complex problem, well almost anybody. The goal is to create a simple design that solves a complex problem. Or failing that, a set of simple components that combined can solve a complex problem.

Lets start with a refresher on how a blockchain works and i am talking about a fully decentralized blockchain, not the ones that have just a few nodes controlling block production, or are editable/censorable by some central authority, etc.

If you havent read the satoshi whitepaper, now is a good time to do so… OK, so what did it say? At first when you realize that all nodes have to keep track of the entire lifetime transaction history, it just seems so impractical. However, we are almost in the year 2020 and what used to require large computers are now possible with most phones. Storage capacity also keeps increasing, so what even in 2009 might have seemed to be large numbers are now not so big after all. The first step to understand that efficiency is traded off for security. It is due to this inherent inefficiency of blockchain that it is of utmost importance to make implementations as efficient as possible.

The consensus code will need to run on every single node in the network for every transaction!

Think about that.

Each line of needless code is inefficiency multiplied by N (number of nodes). The designs therefore need to be made to take advantage of all resources available. In the process of writing the many CC, I developed some useful tricks to allow for efficient processing, but this is for the implementation stage. A bad design, no matter how efficiently implemented will remain a bad design.

Bitcoin validates spends and updates the unspent data. At the core, whenever a transaction is being validated, all its inputs need to be verified they are valid and unspent and that the total spent doesnt exceed the sum of the inputs. And of course, if this is all we are able to tap into, it makes creating anything resembling a “smart contract” quite difficult. Granted, there are opcodes like CLTV (check locktime verify) and hashing opcodes that can verify a secret matches a previously released hash, it is all very low level and how can any arbitary problem be mapped to hashlocks?

It cant.

This was the impetus to add cryptoconditions to the komodo codebase, but after realizing that being able to do a bit more flexible conditions with fulfilments, it became clear that even more was needed. Thus the custom evaluation function was created. This custom evaluation function is passed an otherwise valid transaction and its job is to return true or false. There are no other constraints on it.

That opened the floodgates as now arbitrary C/C++ code could be made part of the transaction validation. Now we have the ability to make any code specification into a CC (custom consensus), but this ability needs some methodical way of being achieved. Since arbitrary C/C++ code is turing complete, by the definition of turing completeness any program that can be implemented can be implemented in C/C++ and as such it can become part of a CC.

The first step to designing a CC is to restate the problem in terms of transactions. You need to transactionalize the problem. What this means is that to change the internal state of the program, a specific transaction is needed. If there are strong incentives for users to make these transactions just from the design, there is no need to have mechanisms that ensure the transactions are done. For example if you send locked funds to a user and they cant spend it without issuing the unlocking transactions, then we can assume the unlock transaction will be done.

In the case there there is no specific person that would have financial motivation to issue a transaction, the solution is to incentivize issuing the transaction with a fee. it becomes similar to mining where people make coins by running software, but this code just needs to monitor new blocks for tx that can be issued, so it is very low resource consumption. even small rewards will be enough to ensure the transactions are made, since people like money. This does assume that the coins have some actual trading value.

Let us imaging a problem that has a starting state and based on various user inputs it runs its algorithm to determine who should get fees. All the actions that happen without user inputs can be bound to a single transaction, so the different paths that are possible end up being constructed by a set of custom transactions, each changing the state of the algorithm. Proper incentives along the way to ensure that the transactions are made.

That is the basic idea, let us illustrate this will a few examples. The simplest example is the faucet. The problem is that we dont want people to be abusing the faucet (too much) and that some amount of coins are released to new users. One thing to keep in mind is that a vout is only validated when it is spent, so you can create invalid vouts and not know it is invalid until it is being spent. What this means is that there needs to be a CC specific vout being spent in order to trigger its validation. The other handy thing is to realize that having a globally known privkey, allows all nodes to properly sign it.

Wait! Did I just say a privkey known to all nodes? Isnt that insecure? Well, certainly if you only relied on the signature of the privkey it would be a mad race for everybody to spend these funds. However, the custom consensus means the validation function must also return true. I cannot stress this point enough, the custom validation will determine the validity of spends, so of course it cant have bugs and it needs to handle all the cases.

These do cause implementation and testing issues and you cant be making assumptions. But these are implementations issues, so lets go back to the design. With komodo chains they have the -addressindex functionality built in, so like an explorer can list all transactions/unspents at an address, so can the komodo chains. With the faucet, we can use this to identify addresses that are new and reject a faucet claim by an address that is already active. The other aspect is that we need to discourage changing to a new address to get more coins, so the solution to that was to limit it to 0.1 coins and to require txpow. The txpow requires the txid to start and end with 0x00. In practice this takes 20 seconds to one minute to calculate with modern systems and spending that much CPU time for 0.1 coins, likely more can be made via mining.

The entire point of the faucet is to deliver coins to new users, under their control. The transactionalization is split into the funding transaction that funds the faucet and the faucetget transaction that users generate to claim their funds. Since the user that wants the faucet coins is incentivized to generate and broadcast the tx, we can be assured that this is a self-running system on the faucet delivery side. We will have to assume that on the faucet funding side, the community or whichever the project the chain is run for will be incentivized enough to fund its faucet.

This faucet example is quite simplistic and it likely doesnt convey the change in paradigm that is needed to transactionalize algorithms. Let us skip to something a lot more complex, like a stable coin via pegs CC. OK, that is indeed a big jump but it shows the power of CC quite well and once you understand the trustless oracles that is the foundation for this, it is not that big of a leap.

Creating a decentralized pegged and backed coin is not an easy problem. Prices are very volatile on the side of the crypto backing the pegged coin, but the pegged coin is supposed to maintain a stable price. Some have said this is just not possible, especially using a decentralized system. However doing the “impossible” is what CC are really good at.

We will assume the trustless oracles work as intended and we have a reliable pricefeed for conversion of crypto to fiat, for this example let us make a USD denominated stable coin backed by KMD, KUSD. At the high level we want the system (blockchain) to issue more USD coins as the price of KMD rises and to burn KUSD coins as the KMD price drops. Ideally the value of the KMD that backs the KUSD coins never goes below some reasonable reserve percentage, let us plug in a value of 20% for the minimum reserve.

We will set the blockchain mining reward to be very low, or make it mostly a PoS chain so the cost to keep the chain mining new blocks is minimal and can be absorbed by community donations. Clearly it cant be issuing large amounts of unbacked KUSD coins from mining! Also, we will be using the one day delayed trustless oracles price to settle the amounts, this means in the event of a catastrophic drop without any recovery, the overall system wont be able to recover. However, if the coin that is backing the system has a catastrophic failure, then it goes to reason that anything that depends on it will also fail. The design needs to handle a bear market cumulative loss of 90%, but we can socialize losses from a catastrophic event that exceeds the capacity of the system to absorb.

So we now have a chain with minimal mining rewards and the goal of creating a stable coin that can handle a 90% loss in value of the backing crypto. In order to allow it to respond faster to large market price changes, the lower of the correlated price or smoothed price will be used. That will allow even a catastrophic loss to be handled as long as it takes longer than 12 hours to unfold.

Looking at the entire supply of issued coins as compared to the price of the backing, we will have an overall reserves ratio. As a system, when this ratio is reduced to 20%, mass liquidations must happen to keep the total system in balance. The requirement to make it a decentralized system means that anybody should be able to issue or burn KUSD coins. The transactionalization would therefore make sense to have an incentivized liquidation transaction where any user would gain instant profits (funded by what is left in the account). This would make accounts near the borderline subject to liquidation risk and the motivation increases for the users to increase their accounts reserve level.

With this high level concept we can now design the entire system. Each user would have their own account tied to their address. New KUSD coins are issued when deposits are made to their account, at up to 80% of the current price, but more practically at 50%. Anything at 80% issued, means only the minimal 20% reserve, so any negative price movement and the entire 20% would be at risk from liquidation. Each user account is monitored independently, but ranked from smallest reserves to largest, with the smallest reserves first to be liquidated when it is needed for KUSD coins to be burned.

The transaction that liquidates an account would be able to spend all the backing coins for the reserve that is left and only need to pay out to the system whatever remains after collecting a 5% arbitrage fee. This 5% can then be converted to immediate profits by selling the KMD on exchanges. The overall reserve rate for the system is increased as the most exposed one was not only eliminated but the excess of 5% was put into the general system reserves.

In the case the price of KMD increases, the equity in peoples accounts increase and the more it increases the more available equity to issue more KUSD coins, which we can assume will be done to hedge against negative price movements, or simply to get liquidity from the KUSD coins. Of course, the ones that are taking out up to the 20% level go first in line for the liquidations.

With both the issuance and burning aspects both realized with a decentralized process, there is just one final aspect to solve, which is the non-liquidation scenario. In this case, it is a normal redeem, which will be done starting with the accounts that have the smallest reserves, but instead of totally liquidating their position, they will get paid a small premium of 1% to 3%. Even though they had the most exposed position and lowering the overall system reserves, since they are within parameters they shouldnt be punished and a small 1% to 3% compensation (based on the actual reserve levels) is meant as a compensation for having their position converted.

Since the redeem size will rarely match any sum of existing positions, each user account acts as a meta-utxo providing inputs to the withdraw transaction. The inputs being supplemented by the one doing the redeeming, and the KMD being sent to them, with whatever change going to the account holders to match their equity plus compensation.

The overall system thus has a self-balancing way to issue KUSD coins when KMD rises, burn USD coins when the KMD price drops and also to allow anybody with KUSD coins to convert them to the KMD backing it. The conversion price might be adjusted a bit to balance out the compensation so that the trading of the KUSD coin achieves as close to 1:1 against USD dollar value as feasible.

Notice the key to achieving this solution was to define transactions that do the critical aspects of issuing, burning and redeeming the KUSD coins. There are many implementation details that are not described, as that is assumed to be something that can be done by studying the source code of the existing CC.

Maybe there needs to be a few more examples, but at least now the extremes are illustrated and I hope this helps you create nice CC designs.