Ethereum Sharding Biweekly Development Update #3 — Prysmatic Labs
In addition to Vitalik’s minimal sharding protocol, there have been lots of discussions about the overall effectiveness and tradeoffs in Ethereum’s sharding vision we will discuss in this new update.
Research, Research, and More Research
One of the main arguments against using a smart contract to mediate consensus across shards in Ethereum is the load capacity of this contract on the main chain. One of the main reasons we cannot grow the number of shards in an unbounded manner is due to the expense this would have on the Sharding Manager Contract. The contract would have too many function calls in a short amount of time.
Moreover, the system has a design constraint known as notary burst overhead. This refers to the idea that when a notary is selected in a certain period, he/she has a deadline to verify as much collation data availability as possible and also vote on available collations effectively. In essence, notaries will need to download a certain x amount of bytes/second, and we aim to minimize this rate for the overall effectiveness of the system. We can calculate this rate as the number of collations a notary has to download * the size of a collation over the time to internal finality.
We can actually modify the other parameters of the equation while increasing collation size and achieve good results, however, it is better to keep collation sizes smaller in practice. The smaller collation bodies are, the faster users can have subjective confidence of their confirmation, as bigger bodies would take longer to confirm.
But how can we fix the dependency on the capacity of the Sharding Manager Contract? Currently, a scheme based on a random beacon chain that is linked to this contract is being explored. This chain, akin to dfinity’s Random Beacon implementation, would be in charge for pseudorandomly sampling notaries and would allow for cool stuff such as off-chain collation headers that were not possible before. Through this, no gas would need to be paid for including collation headers and we can achieve faster finality guarantees, making the system way better than before!
In fact, Vitalik’s latest sharding teaser on Twitter explored this approach via a Python simulation for pseudorandom notary sampling via random beacon chains. This approach would allow each shard to have its own fork choice rule that is tied to the random beacon chain. That is, a collation would point to a parent within a shard and also to a block within the beacon chain.
Our Latest Code
Our Sharding Client’s Recently Merged Pull Requests
Given the possibility of more changes to 1.1 spec, team has been working on the minimal sharding protocol as something it’s worthwhile to develop and less likely to change in the future. Note this protocol is simple and lack slashing condition when notary behaves badly, but still offers a great opportunity to build and test all the base infrastructures for sharding 1.1.
We have merged code to allow notary to register and deregister themselves, proposer can call add header and notary can vote on the header. As a simple workflow, a sampled proposer will:
- Create a collation
- Sign a collation
- Submit a collation by calling add_header from the Sharding Manager Contract
- Then, the collation’s body gets broadcasted via the sharding p2p network
- The samples notary maintains its index after being registered in the SMC
- When a period starts, our sharding client will checks which shard the selected notary can vote for
- The notary then downloads submitted collations and checks for availability
- The notary votes on the first collation header to be submitted that also has a fully available collation body
- The collation gets accepted to the main chain when it reaches quorum size
Here is our latest Sharding Manager Contract source code.
Presentations & Team Updates
Sharding Presentation at the Chicago Ethereum Meetup
Prysmatic Labs Teammate Raul Jordan was recently invited to introduce sharding and its latest research at the Chicago Ethereum Meetup this past week. You can check out the recorded presentation here or see the slides here.
Current & Upcoming Work
Sharding gives us the unique opportunity to explore different technologies we can use for the future of Ethereum’s technology stack. In the next updates, we’ll be exploring shard state local storage via different key-value databases such as leveldb, rocksdb, and more. We will be exploring modifications to Ethereum’s Merkle Patricia tree to improve efficiency in a cross-shard ecosystem as well as benchmarks for disk I/O utilization.
Enrique Fynn’s New Published Paper
Our teammate Enrique Fynn also recently published a paper (arXiv) on the inherent challenges and tradeoffs when partitioning blockchains. Enrique has done a detailed analysis of what would happen if common partitioning techniques from distributed systems literature were applied to the Ethereum state graph as is, and there are some interesting insights you should check out.
The main idea is that Ethereum’s graph was not designed to be partitioned, and naive techniques would lead to a trade-off of either (1) a strong imbalance of state throughout shards, or (2) a large number of cross-links and dependencies across shards.
New Prysmatic Labs Website
Our website is open source, of course. Any issues or suggestions would be greatly appreciated on our website GitHub repository.
Interested in Contributing?
We are always looking for devs interested in helping us out! If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 1 milestone along with a specific project it belongs to (smart contract related tasks, collator node tasks, etc.).
Official, Prysmatic Labs Ether Donation Address
Official, Prysmatic Labs ENS Name