Sharding 102: Takes Two to Tango
Shards are basically disparate blockchains in the same ecosystem whose activities are coordinated by a “leader” chain. The specialization of tasks and independence is the critical driver behind their fast transaction capabilities and scalability. However, what happens when an exchange on the network is made up of two or more transaction types across different shards? How does the network communicate and process this type of exchange?
They Say Hello, They Say Hola, They Say Bonjour
Nodes are responsible for processing transactions, propagating transactions and blocks, and storing network data. For traditional blockchains, nodes will also store the entire history of the ledger and keep adding to it as more blocks are validated. Nodes on sharded blockchain do not store the entire history, rather, the state and history of their shard are what they store unless the shard is the control shard.
Users on a sharded blockchain can have different account balances and status for the same address depending on the transaction they have had on the shard since each shard exists in its own ecosystem.
Transactions on any shard can only be processed by the validators on that shard.
This is known as State Sharding.
Essentially, validators are only concerned with what’s happening locally and will only relay events affecting or emanating from their side, drastically cutting down on computational requirements and resources.
However, when an account on one shard wants to transfer funds to another user on another shard, validators on the sender’s shard wouldn’t be able to credit the receiver account, nor validators on the receiver account capable of debiting the sender’s account.
What this means is neither validators on the two different shards will be able to process the transaction in its entirety alone. They will have to work together to do so.
There are two ways of doing so:
- Synchronous Transaction: In this transaction type, once a node sees a transaction involving a different shard, it queries the other shard for proof to verify the existence of the transaction on that shard and the status of the data required for the transaction. Validators on both shards will need to collaborate together to produce the blocks containing the cross-shard state transition at the same time. However, this process may turn out to be ineffective when processing transactions containing disparate block pairings in a single block.
- Asynchronous Transaction: This transaction type simplifies communication by having one shard execute its end of the transaction before presenting proof of execution for the other shard to execute its own end too. The simplicity of this transaction type makes coordination and implementation a snap, however, there is an inherent risk of blocks getting orphaned if the shards encounter a fork.
Fork it
A fork is any change to a blockchain protocol that alters it from its previous version. This divergence — either due to a software update, consensus change, or rogue miner trying to take over the network — will either splinter the blockchain into two chains or two software versions. The former is known as a Hard Fork, while the later is a Soft Fork.
A Hard Fork is created when a new software version is not compatible with the existing, creating two chains competing for the same computational resources and community. To avoid disruptions and a weakening of the network that can expose it to security risks, agreements between nodes to update their software and start mining the new chain at a predetermined block would have been reached prior to the event. However, not all forks are peaceable.
When irrevocable differences within the community sustaining the chain split the community into different camps, a Contentious Hard Fork may arise as witnessed during the Ethereum DAO hack, whose approximately $55m hack resulted in the creation of a new blockchain to undo the theft or the block size disagreement that birthed Bitcoin Cash. An attacker controlling enough nodes to change the state of previously committed blocks or transactions may also result in a fork.
A Soft Fork is an optional upgrade that maintains some level of compatibility with the previous version. Nodes running the older software will still remain on the same network with those running the newer codebase.
Since a fork will eventually result in the orphaning of a chain, what happens if the finalization of cross-shard blocks encounters a fork? The probability of some blocks getting orphaned, hence irrecoverable, without another fork which may orphan some other blocks is extremely high.
Developing a semi-synchronous model where any cross-shard messages may be delayed up to a certain finite amount of time but it will be delivered eventually resolves this issue. Using this model as a basis for its cross-shard communications, Concordium’s Intershard signaling allows transactions between shards and communication of smart contracts on different shards.
Come Together
Concordium handles cross-shard transactions through communication protocols that allow shards to communicate transactions among each other. Each account/contract is assigned to one shard and every time it sends/receives a transaction from another shard, this transaction is communicated through the communication protocol.
When a block finalizes on a shard, it contains a list of outgoing messages for other shards. The nodes of the sending shard sign the list of outgoing messages. The nodes in the receiving shard can obtain the list of nodes running on the sending shard together with their public keys from the control chain, which allows them to verify the signed messages from the sending shard.
Once a message is verified, it is executed on the receiving shard. For example, a transaction between an account/contract in shard A and an account/contract in shard B that needs to be processed will use the cross-shard communication protocol to send and later acknowledge receipt of this transaction across shards.
The shard committee of a certain shard periodically reports its shard’s latest state (i.e. a block) for finalization to the control chain. The control chain finalizes this state and the shard’s chain is considered finalized up to the block that was finalized in the control chain. However, in the case of an attack that results in the process failing too often, the control committee running the control chain re-elects a new larger committee for the failing shard in order to ensure liveness.
Join Concordium in our different channels:
Twitter: https://twitter.com/ConcordiumNet
Telegram: https://t.me/concordium_official
Reddit: https://www.reddit.com/r/Concordium_Official
Developers Hub: https://development.concordium.com/tech
Gitter: https://gitter.im/concordium_official/community
Discord: https://discord.gg/xWmQ5tp
Learn more: https://concordium.com
