The Yggdrasil Protocol
THORChain’s Unifying Solution to Blockchain’s Scalability Trilemma
One of the biggest obstacles to mass adoption of blockchain technology has been the issue of scalability. This term is defined as the capacity of distributed ledger networks to add users and transaction throughput, without compromising the operability, cost-efficiency, and — in the case of blockchains — security of their platforms.
Herein lies the problem: all participating nodes in a public blockchain network must validate every transaction and reach consensus with their peers. While this mechanism represents the core appeal of distributed ledger technology (DLT) from a security standpoint, it also leads to inevitable bottlenecks as networks grow.
Take Bitcoin and Ethereum, the two largest public blockchain networks in the world, for example. The former can only process three-to-four transactions per second while the latter doesn’t fare much better, with a maximum transaction-processing output of 15 transactions per second. And even more vexing, as user demand for these networks grows, the costlier and slower they become, resulting in a deteriorating user experience.
Compare this performance to centralized credit card network Visa, which leaves both distributed networks in the dust, processing tens of thousands of transactions per second. However, with the advancements in sharding technology, public blockchains may be able to match, if not surpass, the efficiency, reliability and payment bandwidth of legacy credit card payment networks.
Sharding
Pre-dating blockchain technology by over 40 years, sharding is a “method for distributing data across multiple machines,” according to MongoDB, one of the most widely used non-relational database providers in the world. Traditionally, sharding has been a “horizontal” process, where database tables are divided into groups of rows and stored on multiple database nodes, or servers.
In public blockchain ecosystems, network transaction loads are divided into different shards that comprise different participating nodes. Through this arrangement, each node throughout the network processes only a fraction of incoming transactions .
To promote optimal scalability throughout its decentralized ecosystem, THORChain has engineered a unique sharding solution called the Yggdrasil (ɪɡdrəsɪl/) Protocol.
But unlike legacy sharding applications, which reformat databases tables into horizontal rows, the Yggdrasil Protocol deploys a vertical sharding technique based on transactions between entire chains that are stored on different nodes.
How it works
In non-relational, or non-structured query language (SQL) databases, horizontal sharding offers better security at the expense of efficiency and speed. Blockchain sharding proposals work the same way. The basic idea is dividing transaction processing and “global states” — the account balances, contract codes, storage and all additional relevant ledger data — into shards, which are then hosted by different sets of validators.
The Yggdrasil Protocol applies this concept to cross-chain ecosystems, except it vertically shards down entire chains, as opposed to splitting them up into horizontal components. Thus, the protocol leverages sharding techniques to enable multi-chain operability, instead of sharding an existing chain.
This framework is more effective, because it removes the need for a validator to piece together shards. Furthermore, verticalization minimizes cross-shard communication, as transactions are more likely to be assets moving through a single shard.
THORChain engineered Yggdrasil this way to optimize cross-chain token transactions and trading. The protocol enables cross-chain transactions to occur internally within shards or across two shards. This is due to chains being spawned inside of an existing shard first, before being split into their own shard, when prompted by the network. In theory, this model could enable the Yggdrasil Protocol to support 180,000 transactions per second, with less than 10,000 nodes hosting 30 shards.
How it Works
Initially, the Yggdrasil Protocol will have one shard of C number of chains. Let’s say there are 30 chains, for example. These chains correspond to 30 assets tracked on the THORChain ecosystem. However, the saturation of that 30-chain shard is less than 90 percent. Saturation is defined by the block size and benchmark of transactions per second, which in this context, means that there would be insufficient activity on the chain to cross over 90-percent saturation.
But as this chain approaches 90-percent saturation, the shard will split into two shards of 15 chains, with the process repeating itself for every shard that approaches this percentage threshold. Initially, THORChain will design Yggdrasil Protocol so that there are 21 Validators hosting every shard set.
THORChain predetermines each Validator Set to produce blocks on any two shards. The determining number of sets per shards is N choose two. For example, the protocol could have four shards out of which every two is matched, so it would nominate six validator sets to ensure that every shard has a relationship with every other shard.
This model enables three different transaction types:
- Cross-Chain — Entail moving token between two wallets. In this case, the transaction would involve just one chain and one other shard.
- Intra-Shard — Cross-chain transactions that take place within a shard. Validators produce blocks for those two chains and the deterministic producer.
- Cross-Shard — A trade is performed from one chain in shard A to another chain in shard B. For example, User 1 trades their Bitcoin for User 2’s Ether, but because of the nature of this shard split, the Ethereum chain is in a different shard than the Bitcoin chain. However, there is always a validator connecting any two shards. One of those 6 validator sets will observe a cross-shard transactions happening between shard A and shard B, and can produce blocks for shard A and shard B at the same time, and reach finality on those blocks.
Fault Tolerance
THORChain does not engage the full set of validators to achieve shard consensus. In fact, only a minor subset of validators finalize cross-shard transactions, but at least the other 50 percent monitor the transactions, a framework that retrofits the Yggdrasil Protocol with robust fault tolerance — a mechanism design that allows information systems to continue operating properly in the event of failure or malicious network activity.
There are several advantages to this approach. For one, no matter how many shards or validator sets, 50 percent of the protocol always monitors any given shard in a cross-shard transaction, which mitigates the risk of failure.
Secondly, a validator in any particular set does not need to remain aware of all 30 shards.
Instead, validators need only to download and sync two shards out of 30. So, as the network scales its requirements, Yggdrasil Protocol’s nodes and validators sustain themselves continuously. The only thing that prevents the protocol from scaling is the number of nodes and validators that are available to produce blocks.
How Does the Protocol Assign Validators to a Shard? Does that Arrangement Shift Over Time?
The Yggdrasil Protocol assigns 126 validators to THORChain shards through an auction-based process, predicated on how much digital assets, be it Bitcoin, Ether or RUNE (THORChain’s token) they stake on the network.
Then, validators attempt to enter the validator sets that would be most valuable to them. So, the 21 highest stake validators would be hosting data on two of the most valuable shards, because any transactions on those shards will generate the most fees.
Conversely, the validator hosting two of the lowest traded and lowest-valued assets would significantly less transaction fees. It all boils down to cryptoeconomic design. Naturally, the highest staked validators will gravitate to the most valuable shards.
Conclusion
At a 2017 blockchain conference in Taipei, Ethereum co-founder Vitalik Buterin described distributed systems’ prevailing “trilemma” as a tradeoff between scalability, decentralization and security, where only two of three features are capable of co-existing.
Named after the the mythical tree that connected the nine worlds in Norse mythology with three roots of wisdom, the Yggdrasil Protocol’s vertical-sharding solution has the potential to unite these conflicting domains into a cohesive blockchain ecosystem.
Join the Distribution
For more information or to participate in the distribution, head on over to https://thorchain.org/token.
Be sure to check out our official community channels for updates:
- Github: https://github.com/thorchain
- Medium: https://medium.com/thorchain
- Twitter: https://twitter.com/thorchain_org
- Reddit: https://reddit.com/r/thorchain
- Telegram Community: https://t.me/thorchain_org
- Telegram Announcements: https://t.me/thorchain
