Ethereum 2.0 Development Update #31 — Prysmatic Labs
Our biweekly updates written by the entire Prysmatic Labs team on the Ethereum 2.0 roadmap.
Merged Code, Pull Requests, and Issues
Align Runtime to v0.8.1
In the last update we announced to pass test vectors which means the Prysm code base now fully complies with the spec freeze version v0.8.1. In this update we are happy to announce Prysm single node run time is now align with v0.8.1 🎉. This was no easy task, lots more bugs were discovered and resolved to get to this milestone. After resolving a few non trivial run time bugs (#2960, #2961 #2984, #2985, #2986) and with the help of newly implemented eth1 mock rpc, here is a single beacon node running with 64 validators using the minimal set up config. It is important to mention this is the first time we have run time with BLS signature and state root validations fully enabled in a run time set up.
EthereumAPI reference implementation in Prysm
As mentioned in previous posts, we have begun the effort to unify service schema definitions for data collections with the goal to reduce disparity for validator client operators, block explorers and anyone interested in the data of ethereum to access. We have made good progress on implementing these service definitions. Our site api.prylabs.net represents that data a single node would provide. We are excited to live test these RPC services in the test net. The goal as we head toward next week is to relaunch the test net with ethereum apis implemented and sync in a stable state. You can track our progress here.
Template Based Protobuf Parameters
One of the design trade offs made when designing our SSZ library was to hard coded certain ssz information in the field tags of the data structures. For example, go doesn’t have a way to express an artificial maximum size limit for a given slice so we have a `ssz-max` struct tag to indicate the maximum size considered for ssz encoding. Additionally, we needed a way to represent a fixed size array without needlessly allocating N elements. In other words, we needed a slice that was of length `N` for encoding purposes but wasn’t actually allocating `N` memory with empty objects. For this, we have defined the struct tag `ssz-size`.
So we solved that problem, but what about handling different configurations? Now it gets a bit tricky. We’ve hard coded these values at compile time and can’t change them at runtime. This leaves us with two immediate solutions. 1) Redefine each data structure for each configuration. 2) Abstract the structs in with interfaces. The first option is a miserable experience since go does not allow for inheritance or generic structures. The second option is more promising, but interface abstractions tend to lead to more cumbersome code and every value read is a copy of the data so it becomes more expensive on the garbage collector and memory usage.
Neither of those solutions left us feeling very great. So we thought about it a bit more. Since we don’t really need to change the configuration at runtime, we started to think about build time configuration. Using our build tool, Bazel, we were able to invoke compile time template substitution so that we could toggle configuration of these hard coded value without any additional code changes.
The templated variables randao_mixes.size and active_index_roots.size would be substituted and compiled with the appropriate configuration value, be it mainnet or minimal configuration for SSZ encoding. Read more in 3062.
Sparse Merkle Tree Optimizations
One of our bottlenecks that did not manifest in our old testnet but did as soon as we attempted a v0.8.1 testnet run was the creation of our sparse Merkle trie used for deposits and deposit proofs within Prysm, as this was actually taking up a large amount of memory and time once we had a large enough number of validator deposits. We took a quick revamp of the package, made it massively simplified, and benchmarked to ensure it has longevity throughout our project.
Eth1 Mock Server
One of the current focal points of Eth2 implementations is to have some easy, standardized way to mock the “genesis” event of the beacon chain. Many teams are attempting a few different approaches, such as using a yaml file to encode a list of deposits, parse that file, and kickstart their local beacon chain. When approaching this problem, we wanted to ensure our “genesis” event would match the real, production scenario as much as possible. That is, we didn’t want to mock genesis from some file but instead mock it from some set of ETH1 RPC endpoint responses that look like the real thing, with people depositing 32ETH into some contract.
Our current testnet does use a real ETH1 RPC endpoint from the Goerli testnet, with a real deposit contract upon which users can deposit ETH to become validators. Ideally, a solution would manifest in the form of some server endpoint Prysm can point to which can give us mock data that would be enough to start the chain.
We created eth1-mock-rpc for this purpose and it gets the job done by reading a list of validator private keys from a local keystore and serving an RPC websocket and HTTP server. This tool will save us hours and hours of development time, being able to kickstart an almost “real” beacon chain from deposit events received via RPC. We thank the awesome go-ethereum project for providing great utilities for interacting with an Ethereum web3 endpoint in Go itself.
Upcoming Work
SSZ Optimizations
Turns out we weren’t maximizing the usage of our SSZ tree cache for increased performance in our runtime! We spent some time optimizing the cache, making sure it is collision resistant, and overall improving the speed and memory footprint of the go-ssz project itself. Our focus of the project was to achieve correctness first, and optimization second. Given Go does not have support for generics as it currently stands, we needed to make several compromises and find an approach that would make our project easy to test with a standardized API that matches other serialization libraries. There is a lot more work to do on this front and we expect our SSZ project to be highly optimized by our next update.
Database Refactor
One of our goals this quarter is to refactor our db implementation. Currently we use bolt as our underlying key value store. The primary goals of this refactor is to be able to support a larger variety of backends for our db, re-organize packages in the db so that we have proper separation of concerns. Currently we have all the functionality stuffed into a single package, which makes it difficult to read the code from time to time. Also another important goal would be to optimize how we use bolt to persist all the different structures. Currently with how certain methods are structured, those methods take longer to persist. With this refactor any prysm node should be able to run with any backend ex: A key-value store, relational db,etc. This is tracked in this issue and the design is detailed in this document.
Miscellaneous
ETH2 Development Experience Presentation in BuildEth
Our own Terence presented in BuildEth last friday on the topic of ETH2 client development experience. The talk consists of Prysm client architecture, what it takes to translate spec code to production code and how to manage a good open source project implemented from spec. He highlighted the importance of understanding the spec before coding up the spec and it’s best to leverage open source infrastructure out there to facilitate the implementations. The recording has not been posted but you can check out the slides here.
Ethereum 2.0 Implementers Call
We have just completed our bi-weekly ethereum 2.0 implementers call. As research winds down on the beacon chain, majority of the focuses now revolves around researching phase 1, phase 2 and networking challenges. We are excited for the next milestones on multi client interoperability and contribute to research challenges such as light client syncing and execution environment prototyping.
Interested in Contributing?
We are always looking for devs interested in helping us out. If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 0 milestone along with a specific project it belongs to.
As always, follow us on Twitter or join our Discord server and let us know what you want to help with.
Official, Prysmatic Labs Ether Donation Address
0x9B984D5a03980D8dc0a24506c968465424c81DbE
Official, Prysmatic Labs ENS Name
prysmatic.eth
