Ethereum Sharding Biweekly Development Update #2— Prysmatic Labs
So many exciting things to share in this update! We’ll dive into some critical new discussion from the Ethereum Research team and discuss what Prysmatic Labs is working on.
Technical updates
Sharding Phase 1 Specification (Version 1.1)
In a sharded universe, technical updates move fast. By this I mean to say that the Sharding Phase 1 Spec that we mentioned in our last bi-weekly update is already deprecated!
A special thanks to Ben Edgington and Nicolas Liochon for writing concerns about the collator / proposer split which ultimately led the research community to explore a better design. In their ETH Research forum post, they argue that collators and proposers would eventually be operated by the same actor since this scheme allows proposers to verify transactions without maintaining state data by way of stateless client technology in Phase 3. As such, sharding client teams are changing direction to #BUIDL other key features. This includes a sharded p2p network, collation body serialization, local state trie storage, and potentially a new role of notary.
Sharded P2P Network
One proposal for a sharded network by Jannik Luhn recommends using a Torus-shaped sharding network. In this paradigm, there would be a single network that all shards share rather than a network for each shard. Nodes would propagate messages to peers interested in neighboring shards. For example, a node listening on shard 16 would also relay messages for shards in the range of 11 to 21 (i.e. +/- 5). Actors that need to listen on multiple shards or quickly change shards could find peers that may relay necessary messages. A node could potentially have access to messages from all shards with only 10 distinct peers for a 100 shard network.
The team at Prysmatic Labs has been exploring this solution; we’re considering replacing DEVp2p with libp2p framework, which is actively maintained, proven to work with IPFS, and comes with client libraries for Go and Javascript (with Rust, Python, and Java coming soon!).
Collation Body Serialization
The collation body consists of 1MiB of arbitrary data. This blob data will most likely consist of EVM executing transactions, but there is an opportunity for the data to specify that to skip EVM execution. Developers may operate their own execution engine and package their data into shard collations. This proposal for blob serialization by Justin Drake and Preston Van Loon (disclaimer: this is me!) outlines a strategy to partition collation blobs into 32 byte chunks with the leading byte acting as a terminator index and blob flags.
The Prysmatic Labs team has started implementing collation blob serialization for geth-sharding.
Local State Storage
With a sharded network comes sharded state storage… or does it? State sync is already extremely difficult for clients today. While the blockchain data stored on disk might use~80gb for a fast sync, less than 5gb of that disk is state data while state sync accounts for the majority of time spent syncing. As the state grows, this issue will also grow. We imagine that it might be difficult to sync effectively when there are 100 shards and 100 different state tries. As such, Prysmatic Labs is exploring other solutions in close collaboration with the Ethereum Research team. One recommendation from Vitalik Buterin outlines using sparse merkle trees.
Notaries
The concept of a proposer/notary is introduced by Vitalik Buterin in this research post and expanded upon by Justin Drake here. The team at Prysmatic Labs is following this topic closely as there may be some opportunities to add functionality to the Sharding Management Contract in phase 1 despite the plans for tight coupling to come in phase 5.
Donations
A special thanks to SpankChain for contributing $25,000 to Prysmatic Labs!
SpankChain is an economic platform built on the Ethereum blockchain that serves the $100B adult entertainment industry. By using their digital currency wallet and integrating with their payment channel hub, their platform businesses can provide an instant, secure, low-cost payment option to their users. To learn more about SpankChain, read their white paper, watch their introduction video, follow them on Twitter, join them on Discord, and subscribe to their subreddit.
We are extremely grateful to have contributions from companies like SpankChain. These donations will go towards helping our team reach our goals and fund open source initiatives to ultimately scale Ethereum for everyone.
Interested in Contributing?
We are always looking for devs interested in helping us out! If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 1 milestone along with a specific project it belongs to (smart contract related tasks, collator node tasks, etc.).
As always, follow us on Twitter, drop us a line here or on our Gitter chat and let us know what you want to help with — we need all the collaboration we can get! 🎉
Official, Prysmatic Labs Ether Donation Address
References
- Phase 1 Specification (Retired) by Justin Drake
- Exploring the proposer/collator split by Ben Edgington and Nicholas Liochon
- Torus-shaped sharding network by Jannik Luhn
- Blob serialisation by Justin Drake and Preston Van Loon
- Vitalik’s comment on Data Availability Proof-friendly State Tree Transitions
- A general framework of overhead and finality time in sharding, and a proposal by Vitalik Buterin
- Expanding on proposer/notary separation by Justin Drake
