Ethereum Sharding Biweekly Development Update # 11 — Prysmatic Labs
Our biweekly updates written by the entire Prysmatic Labs team and we hope you enjoy them!
Merged Code, Pull Requests, and Issues
Aligned All Our Code With Latest 2.1 Spec
Multiple pull requests have opened to modify our core blockchain package to align with latest 2.1 spec. First, we updated the block and state fields in data structures, then we made minor tweaks on shuffling and cut off algorithms as recommended by Ethereum Foundation Research team, finally we implemented functions on apply Casper FFG rewards and penalties. Second, we implemented new helper functions to get block hash by slot, and validator committees information by slot. Third, we are implementing processing attestation functions. Attestation is the primary load on beacon chain. Every beacon node will process all the attestations coming from block to calculate active and crystallized state. We plan to finish attestation processing function this week then move on to dynasty transition!
Deprecated Sharding Manager Contract, Created Functioning Beacon Client
Originally, we had a lot of contract code related to the Minimal Sharding Protocol in a sub-folder of our repository called client. However, since the beacon chain + Casper research has been underway, its specification would deprecate the Sharding Manager Contract that was originally proposed before the Casper+Sharding announcement.
The purpose of this contract in the Minimal Sharding Spec and our implementation was to:
- Handle the submission of collation headers and proofs of custody across shard, period pairs
- Select committees (notaries) on shards for a certain period
- Reshuffle committees across shards
These pieces of functionality will be replaced by the Random Beacon Chain implementation and its PoS mechanisms + shard cross-link handling. In order to tie in our previous work on the minimal sharding spec with our beacon chain work, we completely updated our /client sub folder to align with the beacon chain spec. Now, our client project spins up a standalone client connected to a beacon node which listens for latest beacon chain events and handles proposals/attestations! Read more in this pull request.
Implementation of Beacon Node gRPC Server Streams
Our running beacon node interacts with a standalone client via the popular gRPC framework developed at Google. Currently, the client listens for incoming, canonical beacon blocks and crystallized states in order to determine when it is time to act as an attester or a proposer. Having a distinction between a gRPC server and client allows for use of the awesome gomock library for feature testing. We implemented the logic of the main gRPC server functions to allow for a beacon node to stream latest canonical chain events to connected clients. Now, users can spin up a client and a beacon node with an enabled simulator flag to see the streams in action!
Initial Sync for Beacon Chain Done
For the beacon chain we have merged the functionality for initial sync in this PR. This allows a node which has no stored state to sync up to the latest processed block so that the local chain head is in sync with the heads of the other chains. This is done by initiating the sync from a block with the latest finalized slot. Then the node sequentially queries for the the other blocks with higher finalized slots till the chain is in sync with the other nodes.
More Kubernetes
We’ve merged in more configuration for a Kubernetes cluster configuration to run a sharded system. Specifically, preliminary config for the beacon chain nodes. While running a full system cluster is farther down in our roadmap, we’re developing more of the configuration as we build on Prysm. This methodology gives us early signals to pain points for node operators. Additionally, we’ll have a cluster framework in place for testing sharding beyond what a single machine can run. Read more on the kubernetes progress in this pull request.
Upcoming Work
Separating initial sync and incoming block sync
Currently, regular sync and initial sync are handled by the sync service. As our implementation grows and becomes more complex we decided that it would be best to split the sync service into two separate services. In our Beacon Chain Sync design doc we explored what this would entail here. With this pull request, we separate the sync service into two separate services where only one or the other can be run at the same time. The initial sync will first check if there is any stored state, if there is none it will begin its sync otherwise it will exit the service. The reverse applies for regular sync, where it will only be run if there is stored state in the db. This way only one service is run at once. This functionality is being implemented in this pull request.
Advancing the Beacon Chain: Fork Choice Rule for Ethereum 2.0
One of the most exciting pieces of functionality for Ethereum 2.0 is the new fork choice rule that will decide a canonical beacon chain. Vitalik has coined this “Immediate message GHOST” as an Eth 2.0 enabled GHOST fork-choice rule (Greediest Heaviest Observed Sub-Tree).
In our current implementation, we treat every newly received block as canonical, but that is quickly changing in our next few pull requests where we keep a DAG of received blocks and states and then apply immediate-message GHOST continually to determine the Beacon Chain. Determining what is canonical will involve checking attestations in the active state corresponding to a block in order to finalize a certain slot number.
P2P Adapter Middleware
We’re working on refactoring our peer to peer incoming message subscriptions by introducing an Adapter pattern to enable middleware to process messages before they are received by the downstream code within our client. This type of functionality allows for new features such as: request rate limiting, incoming message monitoring, attaching request instrumentation, or debug request logging.
In the above example, we have an incoming peer message. Let’s say that it’s a request for a beacon block as part of syncing. The client as two adapters:
1) Increment a request counter for monitoring.
2) Rate limit the user.
All of the adapters in the adapter stack must call the next method in the chain. Otherwise, the request is terminated before it reaches the handler. In the case of a rate limiter, the request would be rejected and terminated before reaching the handler. This pattern allows for the rate limiting logic to be abstracted from the handler and reused for multiple request types. Read more about these changes in the pull request.
Misc
Ethereum Sharding Implementers Call #1
This call had some great discussion from client implementers including Ethereum Foundation, Prysmatic Labs, Lighthouse, Harmony, Pegasys, Nimbus, eWASM, and more. We talked more about p2p network simulations, wire protocol, bootstrapping the validator nodes, and Devcon!
Interested in Contributing?
We are always looking for devs interested in helping us out! If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 1 milestone along with a specific project it belongs to (smart contract related tasks, notary node tasks, etc.).
As always, follow us on Twitter, drop us a line here or on our Gitter chat and let us know what you want to help with — we need all the collaboration we can get 🎉
Official, Prysmatic Labs Ether Donation Address
0x9B984D5a03980D8dc0a24506c968465424c81DbE
Official, Prysmatic Labs ENS Name
prysmatic.eth
