Ethereum 2.0 Development Update #37 — Prysmatic Labs
Our biweekly updates written by the entire Prysmatic Labs team on the Ethereum Serenity roadmap.
Devcon Recap
Prylabs team at Devcon
Some of us at Prysmatic made it over to Osaka, Japan for Devcon V, the world’s biggest Ethereum conference. As always, the content was stellar, with so many of the highest quality talks full of avid listeners, and we were impressed with just how much passion there is to make the Ethereum project last the test of time. With phase 0 mainnet release on the horizon, we were inspired to continue working on improving our public testnet, with our teammate Preston Van Loon giving an awesome talk on how we’ve monitored our current testnet runs and what happens behind the scenes when debugging and improving the Prysm project for production readiness.
Terence Tsao gave the Prysm Q4 update at Osaka Ethereum Lighting Talks, which highlighted new features such as round robin sync, new BLS paring library and various performance improvements that went in over the last month. He highlighted the importance of having a public test net to gather staking feedback from the community early on.
We have a lot more work to do, with exciting developments on phase 2 Eth2 research involving cool WASM magic for smart contract execution on behalf of the Quilt team and the EWASM working group. For now, we keep tightening up our production logic, monitoring, and improving some of the bottlenecks we have found in our codebase over the past few months.
Merged Code, Pull Requests, and Issues
Attestation aggregation fixed
Just before Devcon, we learned that our aggregation algorithm had a fundamental flaw with overlapping aggregation bits. Let’s unpack what that means. Here’s an example with bitfields representing validators in a committee of size 8.
OK - Two single attestations can be aggregated.
attestation_0: b00001000
attestation_1: b01000000
result: b01001000 OK!
OK - Two aggregated attestations with distinct bits (no overlap).
attestation_0: b00011010
attestation_1: b01000001
result: b01011011 OK!
NOT OK - Two aggregated attestations with same signature in both.
attestation_0: b00000101
attestation_1: b00000011
result: cannot verify signature!In the third example, the least significant bits are overlapping between the two attestations and therefore cannot be aggregated! This a BLS signature aggregation drawback where the signature can no longer be verified if overlapping signatures are aggregated. Our flaw was that we assumed we could aggregate all attestations with the same data root and we would discard any aggregations the failed. This meant we excluded overlapping attestations from inclusion in beacon blocks! We believe this may have been causing issues with finality as indicated in the chart below. This chart should be a nice sawtooth of 0 and 1 representing changes in samples of the current finalized epoch. A value of 2 or more represents an epoch skipped in finality.
Epoch processing optimizations
We have optimized epoch processing to cache precomputed values such as 1) total attested balances and 2.) validator’s vote stats. It eliminates the need to recompute those values at run which was majority of the bottle neck. The optimizations is done via 3 iterations over active validators for basic info, then over pending attestations to collect vote status and finally over validator’s basic info to compute for balance. This shortens epoch processing run time to O(V+A) where V is the length of validator registry and A is the length of pending attestations in state. Credit to Michael Sproul for presenting this optimizing at this Devcon talk.
Bug fix round-up
Despite the lack of wifi at Devcon, the team was busy resolving critical bugs in the last two weeks! The team and other contributors added 41 PRs during and since Devcon. Among these merges, we would like to highlight logging and general improvements from Jim McDonald. Thanks Jim!
Upcoming Work
Phase 2 exploration
Will Villanueva put together at excellent talk on exploring Eth2 phase 2 at Devcon, which is the point when smart contracts become fully functional and Eth2 achieves its full vision in production. If you haven’t read his post on phase 2, please take a look here and bookmark it — as it is an incredible reference for anyone interested in the future of smart contracts. Will and the EWASM team put together a project called Scout, which prototypes WASM execution environments for Ethereum smart contracts in Rust and allows for understanding just how well this idea could work in a real beacon chain deployment.
One of our awesome contributors, Skillful Alex, decided to take up the scout project in Golang itself, creating a repository that runs compiled WASM execution environments in Go using the great wasmer project here. It is currently a proof of concept but is quickly evolving to become a more robust example of what Go can currently do with its WASM support and the toolkit that comes with it.
SSZ mainnet readiness
Simple Serialize, the serialization library used by Eth2, has been one of our biggest bottlenecks for a very long time, preventing us from scaling the number of validators we run in our testnet. In particular, Eth2 uses Merkle roots of objects as keys across various data structures, and Merkle tries lend themselves to nice caching and optimization strategies to prevent expensive recomputation of a root. There are 3 major optimizations we can do to our SSZ library:
- Precompute sizes of the marshaled eth2 data structures so ssz does not need to infer it from objects directly
- Prevent usage of intensive, recursive reflect function calls to determine the type graph of an object by providing precomputed instances of eth2 data structures at compile time
- Prevent recomputation of Merkle roots if only a few objects changed in a data structure by smarter caching strategies at the Merkleization level
The lack of number 3 in particular has been the reason our runtime has not been scalable when it comes to serializing data. A single object changing in an array of roots in the beacon state would cause the entire hash tree root of the beacon state to be recomputed. We started a PR for this optimization here you can follow along. We are confident once this is merged in, finding the merkle root of a mainnet beacon state in our runtime will be lightning fast.
