The Series: Quadrans Essentials on what is Sharding and how does it Apply to Blockchain?
Hey there, happy to see you again! Since the beginning of the Series: Quadrans Essentials we have walked you through most of the underlying notions of blockchain technology. How is your journey going?
To refresh your memory, here are the topics we have covered so far: Peer-to-Peer Networks, Post-Quantum Cryptography, Public-Key Cryptography, and Crypto-Puzzles Algorithms.
A special episode was also released to highlight one of the features that make the Quadrans blockchain unique, being Quantum-proof Ready.
Today it’s time to tackle a more complex topic and get a little more technical. Let’s talk about Sharding and how it can help the blockchain become more scalable and process more information per unit of time.
ELI5 (Explain Like I’m 5). What is Sharding?
You head to the library to satisfy your desire to read “Lord of the Rings” and ask the librarian to go fetch the book for you.
Meanwhile, a queue of other customers is forming behind you — each with their own book request. Only they have to wait for the librarian to come back before being served.
One librarian to satisfy a bunch of customers is not ideal: he wastes too much energy and time to go back and forth between the shelves and cannot perform multiple tasks at the same time.
To serve customers more quickly and handle their requests more effectively, the library decides to hire new staff.
Few librarians now run back and forth between the shelves of the library, search for the required books and bring them to the customers.
With more staff, the system has improved — but efficiency still has a long way to go… and customers and requests still pile up.
The library is immense, why not divide it into subsets and give a specific task to each librarian? One deals only with romance books, one with mystery books, and one with fantasy books.
“Lord of the Rings?” No problem, just ask the librarian assigned to fantasy books and he will find it in no time.
With this system, the queue of customers is processed much faster, the energy required to search for books is used more efficiently and requests are resolved more quickly.
Well, the library just applied Sharding to improve the performance of its book system.
We can think of Sharding as the process of dividing a single large task into smaller tasks, so that they can be processed with a higher degree of parallelization and efficiency (energy and time wise).
In a nutshell: sharding improves scalability!
However, not all that glitters is gold. What’s the dark side of Sharding?
Sharding comes with certain drawbacks and needs to be implemented carefully.
First of all, finding your way around the different sharding techniques and choosing which one to use is no easy task. The characteristics of the specific target dataset must be analysed correctly.
Also, be prepared to handle more complexity in the system.
If the correct technique is not identified or the complexity in the system is beyond your capabilities, sharding ‘optimizations’ may lead to even worse results than the non-optimized case, resulting in a degenerative scenario.
Sharding and Blockchain Technology
For blockchain technology to really be considered for mass adoption and applications in different industries, it needs to become much more scalable.
It is a fact that over the past year and a half, sharding techniques have been implemented even in complex decentralised infrastructures such as public blockchains to tackle scalability issues.
How does this process work?
Splitting a blockchain into shards means processing more transactions in parallel and making nodes responsible for one shard at a time instead of the entire blockchain, greatly reducing hardware and storage requirements.
Imagine the Blockchain infrastructure as a Big City in which citizens (nodes) and marketplaces (dApp) are spread across a vast space. To exchange information (transactions) and access marketplaces, citizens needs to travel the city, but them walking the same streets or entering the same marketplaces can create very long queues.
To speed things up, the structure of the Big City is divided into Districts to enclose in the same space the citizens and marketplaces they interact with the most. This means that some parties we rarely interact with are not available in our District, but we can always go out of one District and do our business in another, the important thing is that there is a trace of all the events happening in the Big City somewhere. This in blockchain is made possible by the distributed consensus algorithm.
This is in a nutshell is how sharding works for blockchains. But what about security?
If the integrity of a blockchain is built on the concatenation of information, organized over time into blocks by a hash mechanism and secured by the distributed consensus algorithm (Proof-of-Work) — can splitting the chain (sharding) break this system and make the blockchain less secure?
Yes, we are heading straight for Buterin’s Trilemma: scalability, security, and decentralization on the vertices of a triangle that prevents a single solution from finding the “best” value for each of the three elements simultaneously.
Sharding would therefore benefit blockchain scalability but threaten security. The game is to find an appropriate balance.
Let’s go back to our example, the blockchain is a Big City divided into Districts (shards) and filled with citizens (nodes) and marketplaces (dApps). In this scenario, who is responsible for the information flowing within the Districts? And who verifies the accuracy of the information traveling from one District to another?
Leaving information theory (IT) to those who enjoy black screens and green text, let’s try a more practical approach here.
To ensure the integrity of the information exchanged, the citizens periodically elect, among a set of eligible candidates, a High-level Government, made of the so called validators, with the purpose of overseeing the operations of all the individual Districts and keeping track of the status of the Big City. High-Level government then elects Low-level governments for each District.
Only the High-level government will store and check the correctness of each piece of information being exchanged within the Big City, while Low-level governments will only deal with the information of their own District.
Unfortunately, our Big City is not safe from “corruption”.
What happens if one District tries to corrupt another and take over? To avoid this unfortunate but real scenario, the validators elect Low-level governments periodically and randomly, so that each government cannot know in advance which District it will govern and which its colleagues.
This solution offers good scalability and, thanks to its multi-level control structure, improves the degree of security compared to a single-government solution.
By the principles of Buterin’s Trilemma, if scalability and security increase decentralization will inevitably go down. If we consider a traditional Proof-of-Work blockchain, in which all nodes compete with each other, with sharding only a subset of nodes will compete for validation — decreasing the decentralization of the entire ecosystem.
What about Quadrans?
In the Quadrans 2.0 Yellow Paper, Sharding was explored and leveraged as a means to increase infrastructure scalability.
Transactions exchanged by actors in the P2P network are divided into shards based on the addresses of the senders. Specifically, a defined portion of the addresses (the initial N bytes) is used as a grouping key to divide M addresses into N different shards, where the respective transactions will be processed.
The Quadrans network will support two types of control entities:
- Miners, similar to Low-level governors, are in charge of validating blocks within a single shard-chain (District) using the PoW mechanism;
- Masternodes, similar to the High-Level governors, are in charge of controlling the Miner’s work, electing them and adding information within the master-chain.
A node can be eligible for a Miner or Masternode role according to the capital (QDTs) that it keeps locked into its wallet — a miner needs only 1k QDTs, whereas a Masternode needs at least 100k QDTs.
Both Miners and Masternodes are rewarded with new Quadrans Coins (QDCs) in proportion to their role and contribution to the network.
The transactions stored and secured by the Miners in each shard are collected and validated by the Masternodes in each block, compressed into a compact data structure called the Merkle-Tree and added to the master-chain.
Maintaining consistency on all updates occurring on a specific address between shards is essential.
In addition to the master-chain and the different shard-chains, Quadrans 2.0 features a third type of chain, called the synchro-chain, used to mark time.
Synchro-chain, divides time into epochs, and defines for each of them:
- The set of active Masternodes;
- The set of Miners assigned to each shard that compete in the consensus mechanism of their respective shards;
- The number of active shards during such epoch (computed on the basis of a post-analysis of the workload assigned to shards of previous epochs);
- The duration of the epoch.
We don’t want to put too much on your plate guys, so that’s it for today!
This important episode includes complex concepts that hopefully we have made as simple as possible for your understanding. Our journey continues with the next episode of the Series: Quadrans Essentials in which we will see in detail Quadrans’ consensus and validation protocol. Stay tuned and thanks for following us!
