Ethereum Sharding Biweekly Development Update #8 — Prysmatic Labs
Latest Research Updates
Research Notes Leading Up to the Beacon Chain
The merging of Casper and Sharding were major news that took the community by storm quite recently, but the research leading up to the idea of a random beacon chain guiding the development of Ethereum 2.0 was nothing new.
When handling the sharded system via smart contract on the Ethereum mainchain, we were able to derive implicit finality via the transactions that submit a collation header into the contract that would then be mined onto a block in the mainchain. However, we were bounded by gas and the current functioning of the EVM 1.0. That is, the number of shards realistically could only grow as much as the sharding manager contract could handle as a load of incoming transactions.
This system is also more affected by hard-forks and changes occurring in the main Ethereum network. Moreover, the incoming integration of a hybrid Casper PoS system would create a complicated co-existence of two types of validators: namely Casper and sharding validators. Signature verification, in particular, is an extremely expensive operation if done entirely through a smart contract on the mainchain, creating inherent bottlenecks in a hybrid system.
Instead, the research proposes the creation of a sidechain known as a beacon chain that has links to the mainchain by containing hashes of canonical mainchain blocks within its own block construction.
One idea that naturally arose from the research was to store collation headers off-chain, allowing for more shards to exist as there would be no processing bottleneck and faster finality guarantees. Reminiscent of DFINITY’s chain, this process could be done through a construct known as a random beacon side chain that is pegged to the main chain via checkpoints on main chain blocks.
Reminiscent of plasma chains without the exit mechanism according to Justin Drake in his ETHResearch post where the beacon chain was first mentioned, this construct would provide a solid ground for experimentation of shard designs without modifications to the main chain. The beacon chain would provide the pseudorandomness required for committee selection of the sharding system, through BLS distributed key generation, as well as better finality given that shards only care about finalized deposits from the main chain.
Management and submission of collation headers could now be done within each respective shard, with the beacon chain only serving as a coordination device for summarizing what was voted on within each shard. That is, the beacon chain would be responsible for handling what we call a cross-link, which is a piece of metadata summarizing which collations were voted on as canonical within shards, who voted on these collations, and the merkle roots of their data.
We have put together a document summarizing all the research notes leading up to the creation of the beacon chain here where you can follow along.
How Are We Designing the Beacon Chain
Since moving away from the Sharding Manager Contract (SMC), the team has been thinking what’s the most efficient way to replace the SMC with a minimal viable beacon chain and merge aspects of our current work in geth-sharding into this new beacon chain that we can use for demonstration purposes. We started a beacon chain design doc to summarize our thoughts. We liked the cleaner approach of having a fresh repository with no dependencies with Geth. Such approach gives us the flexibility to have a better test suite, a leaner architecture, and allows us to work with a clean slate upon which we can establish good practices. We can also use tools such as bazel as part of our build system. In the beacon chain design doc, we summarized the near term goals of the different responsibilities of actors in the beacon chain architecture and we determined the basic components that we need for a successful MVP and which ones we can push out until later.
In the beacon chain design doc, we summarized the following beacon chain components:
- Shuffling for validators
- Advancing beacon chain blocks
- State transitions in the beacon chain
- Fork choice rule for beacon chain
Also, we went over the different actors that participate in the random beacon chain:
- Beacon chain node for listening main chain block checkpoints via RPC connection to running geth node
- Proposer for creating blocks for beacon chain and shard chain
- Notary for signing cross link
- Attester for attesting blocks for beacon chain and shard chain
We listed out the work estimates that’s needed to complete this minimal viable beacon chain. Each work estimate will be transferred to become its own github issue. In the second part of the design doc, we touched base on how we can create a new system without dependencies on geth into a monorepo that will contain both our beacon chain, our smart contracts, and all the code relevant to the sharding client for Ethereum 2.0.
Current & Upcoming work
Removing go-ethereum Codebase and Introducing Bazel
When we started this project as a fork of go-ethereum, we wanted to maintain the ability to merge upstream with the original go-ethereum client. As we advance farther in research and implementation, it is becoming clear that it will be difficult to merge back with the main repository. Sharding is Ethereum’s opportunity to introduce solutions that are considered to be breaking changes. Building our project in a structure that we dictate will allow our team and external contributors to work with higher velocity. We can minimize cognitive effort when deciding where to put new modules and how it can be compatible with the legacy system. The direction and organization of our project is more clear.
As part of this refactoring, we’ve decided to take advantage of Google’s incremental build system: Bazel. Bazel allows you to define make-like definitions called BUILD files to declare buildable libraries and binary executables. A key motivation for this change is that the builds will be fast and reproducible across developer platforms. Not only does Bazel vendor libraries at a specific commit hash, but it also builds with a project specific golang version. Our project is set to use 1.10 and bazel will download this as part of the toolchain. You don’t even need to have that specific version installed on your machine for Bazel to build with it. This means that developers won’t be blocked by testing out the latest branches to go nor will the the blocked by their refusal to update their system.
In addition to the benefits outlined above, we also chose Bazel because of its optimistic future for large projects. The Bazel team is working on remote execution of builds and remote caching of build actions. This can significantly decrease build times across workstations and our continuous integration runners. Lastly, Bazel uses merkle trees to determine the hashes for the incremental build actions. As we bow down to Ralph Merkle, we are reassured in our choice to use Bazel as our build system.
Beacon Chain Code
We have started work in our beacon chain repo. This repo, however, will be merged soon with geth-sharding, so make sure to follow geth-sharding for updates on Github. What we have accomplished so far:
- A validator registration contract that lives on the main chain. It lets the validator burns 32 ETH after validator has inputted its pub key and randao commitment. Once the transaction is included in the block then user is added to the current validator set. The 32 ETH can later be withdrawal on the shard chain
- Beacon chain basic type structs for blocks and states
- Beacon chain config values
- Last but not least, we have integrated Bazel with beacon chain repo. Yay!
Please check out the issues page to see if there’s anything interest you. We are always looking for help!
Integrating the Beacon Chain with the Geth-Sharding Repo
Currently as we have since just started on the beacon chain design, we will be implementing the RBC as no ops in our geth-sharding repo. This is so as to allow us to work on both the RBC repo and the geth-sharding repo. By swapping out the SMC with no-ops it unblocks us from just solely focusing on just the RBC as now we can work on both geth-sharding and RBC concurrently. Then when the RBC has been designed and implemented we swap the no-ops and define the interactions with the RBC in geth-sharding.
We will not be depending on geth as instead of having our sharding system as a sub-command of geth we will be having it dependant on a sidechain through a random beacon chain construct. The only reason we would need geth was for a one time deployment of the validator management contract. This can easily be achieved using a rpc connection to a remote geth node.
This allows us to create a new system with no dependency on geth, which has allowed us to migrate to a new build system, Bazel. Currently we still have some dependencies on geth in our code that imports a few packages from go-ethereum that are needed for our sharding client
Libp2p Work for Ruby Release
We’ve started on our p2p API framework and a primitive implementation for local client discovery and communication. Our upcoming Ruby release involves running a sharded system on a local networked series of nodes communicating via libp2p will enable us to demonstrate the actors and mechanisms of sharding. Check out the in-review pull request here. The next steps are to design our discovery mechanisms with other teams, unify our messages, and optimize p2p communication performance for a testnet launch.
A New Face
We are thrilled to add a new core contributor to the Prysmatic Labs team. Please welcome Yutaro Mori! In his own words:
Hello world! My name’s Yutaro, I’m a software engineer currently living in Tokyo, and I’m extremely excited to join Prysmatic Labs! Ever since I saw Vitalik wear a unicorn-llama riding cat as a shirt I knew that Ethereum was the one and I haven’t looked back since. Sharding is a critical project for the future of Ethereum and I hope that I can contribute in a positive and meaningful way.
Please check out our contribution guidelines to see how you can get involved. It includes comprehensive information on beacon chain and sharding for both part-time and core contributors to the project.
Interested in Contributing?
We are always looking for devs interested in helping us out! If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 1 milestone along with a specific project it belongs to (smart contract related tasks, notary node tasks, etc.).
As always, follow us on Twitter, drop us a line here or on our Gitter chat and let us know what you want to help with — we need all the collaboration we can get 🎉
Official, Prysmatic Labs Ether Donation Address
0x9B984D5a03980D8dc0a24506c968465424c81DbE
Official, Prysmatic Labs ENS Name
prysmatic.eth
