Sharding
An old technology used in a new way.
For many people, the crypto and blockchain sphere can be very intimidating. The never-ending technical terms and jargon (consensus, POS, nodes, hashes, digital signatures, aggregate signatures, smart contracts…etc.), the weird number-codes of hashes and wallet addresses, and the lack of comparisons to a everyday life make it seem like blockchain concepts are as difficult as rocket science.
One of the best kept secrets within the blockchain and crypto industry is that most of the technology we use is not new or unique or overly complicated- it’s just pretty simple computer concepts used in new ways.
A recent computer science process being revived and used in a new way is sharding. Sharding has been a part of traditional computer databases for many years, and has been talked about frequently in the cryptocurrency community. Many projects (including Ethereum and Zilliqa) are starting to apply the technique to solve blockchain scalability issues.
In this article, we’ll discuss what sharding is, problems it can solve in cryptocurrency and blockchain, and the unique implementation that FAB has created.
What is Sharding?
Sharding is a type of partitioning of a database that separates a large database into smaller and faster parts called data shards. Sharding, also called horizontal partitioning, makes data storage and look up faster and easier to manage as the data shards don’t share data between each other. This is because as the size of a database and the number of transactions per second made on the database increases, the response time for querying the database increases exponentially. Basically, sharding increases scalability, speed and latency of a database.
A simple example of data sharding would be separating customer data into different servers based on their global location. Customers in North America can be placed on one server, while customers in Asia are placed on another.
Sharding in Blockchain
Sharding in the blockchain is generally a bit more complicated than our customer data example, but it is used to solve the same issues: scalability, speed and latency. Blockchain is all about transferring/ transacting value and data in a decentralized way, taking out intermediaries in the process.
Big proponents of the technology see that it has great potential to reshape the world of business and finance, but currently the transaction throughput of blockchains are quite low. Traditional blockchains like Bitcoin and Ethereum use the Proof of Work (POW) consensus mechanism because it is the most secure consensus mechanism since every node within the network stores a copy of all transactions on the blockchain. This consideration takes precedent. But POW is also the most resource intensive consensus mechanism, resulting in transaction speeds anywhere from 7 transactions per second to a few thousand per second depending on the blockchain. This is not anywhere close to the speed of traditional payment processors like Visa.
Additionally, for the blockchains like Ethereum that allow for the creation of decentralized applications, applications built on the platform cannot scale up to larger implementations because of the scalability issue. As more transactions are done in the network, the worse the applications perform.
On a blockchain network, nodes would be sharded according to any categorization method, similar to how countries are divided into states or provinces for more efficient governance. Transactions would be processed by miners within a shard group a node belongs to instead of the miners of the entire network. This way a lot of transactions will get validated at the same time on the network. This parallel processing improves the performance of the system. Sharding makes storage of all transactions on every node unnecessary, which improves scalability. Because not all of the nodes are involved in the consensus mechanism, it ceases to be POW.
Blockchain networks such as Ethereum looking to implement sharding solutions around Proof of Stake consensus (POS). In POS, specific nodes are responsible for transaction validation. These ‘stakers’ stake some of their value (cryptocurrency) for the ability to validate. The more they stake, the more transactions they are able to validate and receive transaction fees from. The reason many blockchains like Ethereum are looking into transitioning to POS consensus is because it eliminates the resource-intensive mining of POW, improving the transaction speed , and it allows for sharding of the network nodes, improving scalability.
Sharding in this form on the blockchain does have some potential drawbacks. Firstly, inter-shard communication would be needed when nodes in different shard groups need to interact; creating and standardizing this communication method at a scalable speed is not simple. Secondly, POS consensus is not as safe as POW as it puts transaction control in the hands a numbered amount of stakers. Lastly, sharding will not speed up the transaction capabilities of the Ethereum network enough to be sufficiently scalable for enterprise applications.
Because of the security concerns, many new blockchain projects continue to be built with POW consensus and use different techniques such as adjusting block size to improve speed of the network. The scalability problem is still looking to be conquered by many.
Sharding on FAB
A new form of sharding implementation is currently being run by FAB. FAB recently completed a hard fork which introduced many new innovations, including a working POW blockchain that is actually scalable.
Because the network is scalable without sharding being the central proponent, FAB’s implementation of sharding is quite different to other examples based on the uniqueness of the networks properties.
FAB has a main Foundation Chain that runs with a POW consensus mechanism along with Annex chains (side chains) for quicker settlement. These side chains are for specific applications or uses (for example, EBay could run an Annex Chain where all transactions happen on their chain for quicker settlement). Transactions within blocks on the Annex Chain are then packaged together and sent to the Foundation Chain as one whole hashed transaction number. Annex Chains are created to take the transaction burden off of the main chain to speed up scalability and transaction speed.
FAB uses an innovative protocol called Kanban to keep the Annex Chains decentralized. Kanban is a separate network of nodes, ran through the computer’s GPU, that confirm all transactions happening on the Annex Chain; any node within the full FAB network can be a Kanban node. Kanban produces a separate transaction history from the Annex Chain and confirms that the hashed transaction number the Annex Chain sends to the Foundation Chain matches the same hash the Kanban network determines, keeping the Annex Chain accurate and decentralized.
In this blockchain configuration, the Kanban network will be sharded, allowing a smaller number of nodes to be needed to act as ‘stakers’. The process of using a Annex Chain along with a sharded Kanban lessons the security concerns associated with sharding while providing the scalability benefit of sharding.
