Ethereum 2.0 Development Update #41 — Prysmatic Labs
Testnet Updates
Etherscan Launches Support for the Prysm Testnet
In this final update from 2019, we are pleased to announce the biggest block explorer for Ethereum, https://beacon.etherscan.io, has launched official support for Eth2.0 using the Prysm API for our currently running test network. This is a milestone which further brings this project to fruition by giving users a feel for how their validators can perform in the network, as well as having a broader understanding of how rewards/penalties are doing based on validator activity. It is a great first step in understanding how a network evolves and through it, we have already found several important bugs that only manifest at a certain scale.
We’re thrilled to see Etherscan and other explorers evolve to make it easier to see what’s happening in the eth2 world, and this little peek into the future makes it feel a lot more tangible for everyone involved.
Greatly Improved Network Stability
Something that felt previously imaginable was having a long-running, stable testnet release, but after many months of work, we are very satisfied with the result. The maintenance we have had to perform on the network has been slowing down significantly thanks to a wide array of improvements. Most notably, one of our contributors, Jim McDonald, has been investigating a lot of p2p related failures and annoyances our nodes faced and took on a personal task of resolving them. Jim noticed there were many nodes would spam p2p disconnects/reconnects between each other, and that the way we handled the peer lifecycle + the peerstore could be revamped.
Ever since we merged in Jim’s PR here, we’d had little to no problems with the running test network.
Merged Code, Pull Requests, and Issues
Syncing at 100 Blocks Per Second
Although the final, clear bottleneck for syncing is disk I/O, there were a ton of low hanging fruit we were able to take care of last week to speed up from an average of 15–20 blocks per second to over 100 consistently. The key to this was understanding the assumptions we’re making during initial chain sync vs. regular chain sync. In particular, a lot of data can live in memory during initial sync, as our node is not yet participating in consensus and persisting every detail of the state to disk is not as critical. Instead, our teammate Terence worked on a key improvement which got us up to 60 blocks per second by caching the head state as we sync and only persisting it once we finish the initial sync process. Additionally, we would do a massive number of writes to disk for the head block root as the fork choice rule would run during sync, and this is absolutely not needed. Additionally, having a new, custom Merkle root function for the beacon state in Prysm was able to push us over the 90 blocks per second mark. Finally, with a few other small details we saw on the flame graph for syncing, we were able to consistently break 100 blocks per second, proving there is always room for optimization and we’re confident this isn’t the limit yet.
Lightning Fast State Root Function Merged Into Prysm
Being able to determine the Merkleization of the eth2 beacon state is critical for having a concise, 32 byte summary of the “eth2 world”. This state root is put into blocks on the beacon chain and used for critical communications between nodes, ensuring everyone is on the same state. Given this is such a frequent operation and how the eth2 state is such a big object, we needed a solution that’s ready for mainnet compared to our current approach. Determining Merkle roots of data structures was previously done through our generic implementation of the official serialization specification for eth2 known as Simple Serialize (SSZ).
The problem with this is, well, that it’s generic and Go does not have support for generics. Instead, we have to do a ton of type assertions and inefficient operations to get this to work. Instead, we can implement a highly specific approach for Prysm’s beacon chain state that knows exactly how to navigate the data structures, serialize its fields, and compute the Merkle tree over those fields as leaves appropriately. This is almost a 10x improvement from the current approach, also allowing us to have highly specialized caches for certain fields of the beacon state such as the list of historical block roots.
If we add a root to this field, we should only recompute the branch which changed up the trie instead of performing the entire computation once more. Previously, we used to do this and it would lead to very slow state transitions, preventing us from scaling up the validator count. Now, we are a lot more confident in this aspect of Prysm and can scale up to 100 blocks per second during sync.
Fuzz-Testing Caches
One strategy that Prysm uses to increase performance of expensive operations is to utilize a cache of some kind. This process must be done carefully as a corrupt cache is just as bad as an incorrect implementation. We’ve had one too many hasty caching features roll out to know we’d better be 100% sure the first time! A great way for us to have maximum confidence is to apply randomized input based fuzz testing with a method that uses caches and a method that doesn’t use caches. If these two methods return different outputs then we know we have a problem! So far we have applied this technique to our beacon state ssz hashing and found several issues with our cache implementation as well as the non cache implementation.
Since the input data is randomized, we now have much better testing coverage over less used code paths.
Graffiti, State Pruning, and Other Bug Fixes
We welcome one of our new contributors, Leo (known as metanull-operator), in adding a super useful feature to Prysm validators, giving us the ability to add a custom “graffiti” object to beacon blocks. This is a 32 byte field that can be used for anything, such as adding your name, your loved one’s name, an important hash of some real world data, or more. Block explorers already support UTF-8 string decoding for graffiti, with some blocks already containing data such as metanull’s name in a block here.
You can also annotate your proposed beacon blocks using the ` — graffiti` flag followed by a string when running the Prysm validator client. For more instructions on running Prysm and participating in our testnet, check out our documentation on Gitbook here.
Aside from the graffiti flag, the past 2 weeks have seen an increasingly larger amount of bug fixes thanks to help from the community and block explorers such as Etherscan and Bitfly. We’ve managed to not only eliminate bad code, but harden important pieces of core functionality through more strict test cases and validation.
Fully Implement Ethereum APIs
As we’ve mentioned in previous posts, we maintain a set of documented, carefully designed APIs for eth2 in our Ethereum APIs repository here. Although we’ve maintained a relatively small set of endpoints, Prysm did not fully implement the API, instead having some internal endpoints for certain validator <> beacon node interactions. A few users would note several methods from https://api.prylabs.network were missing from Prysm, and at that point we realized we really need to utilize the public API to its full extent. In particular, we were missing a full implementation of the validator service, including functions such as fetching a validator’s performance, retrieving its duties, etc. As part of our v0.9.2 testnet relaunch, Prysm will be fully compliant with Ethereum APIs, making it even easier for block explorers and users of our testnet to understand how to interact with nodes and validators in the network.
Upcoming Work
v0.9.3.2 and Scheduled Testnet Restart
The Ethereum research team has been hard at work on making sure phase 0 is as robust as possible. They have released many improvements and edge-case fixes to the beacon chain spec which aren’t currently reflected in our testnet, as some of them are consensus breaking. That is, we can’t push out these fixes today without restarting the network from scratch or via a hard fork, and given we have a lot of users and data on our current testnet, we want to do this ahead of time via a scheduled restart announcement. Due to the holiday season, this won’t be until early January, but our new testnet release will be stronger than ever, preventing common bugs users have experienced up to this point and improving the fork choice rule for the eth2 beacon chain.
Interested in Contributing?
We are always looking for devs interested in helping us out. If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 0 milestone along with a specific project it belongs to.
As always, follow us on Twitter or join our Discord server and let us know what you want to help with.
Official, Prysmatic Labs Ether Donation Address
0x9B984D5a03980D8dc0a24506c968465424c81DbE
Official, Prysmatic Labs ENS Name
prysmatic.eth
