Ethereum 2.0 Development Update #31 — Prysmatic Labs

Terence Tsao
Jul 25, 2019 · 7 min read
Image for post
Image for post

Our biweekly updates written by the entire Prysmatic Labs team on the Ethereum 2.0 roadmap.

Merged Code, Pull Requests, and Issues

Align Runtime to v0.8.1

In the last update we announced to pass test vectors which means the Prysm code base now fully complies with the spec freeze version v0.8.1. In this update we are happy to announce Prysm single node run time is now align with v0.8.1 🎉. This was no easy task, lots more bugs were discovered and resolved to get to this milestone. After resolving a few non trivial run time bugs (#2960, #2961 #2984, #2985, #2986) and with the help of newly implemented eth1 mock rpc, here is a single beacon node running with 64 validators using the minimal set up config. It is important to mention this is the first time we have run time with BLS signature and state root validations fully enabled in a run time set up.

Image for post
Image for post
Beacon node state trantion on slot 633–635

EthereumAPI reference implementation in Prysm

As mentioned in previous posts, we have begun the effort to unify service schema definitions for data collections with the goal to reduce disparity for validator client operators, block explorers and anyone interested in the data of ethereum to access. We have made good progress on implementing these service definitions. Our site represents that data a single node would provide. We are excited to live test these RPC services in the test net. The goal as we head toward next week is to relaunch the test net with ethereum apis implemented and sync in a stable state. You can track our progress here.

Image for post
Image for post

Template Based Protobuf Parameters

One of the design trade offs made when designing our SSZ library was to hard coded certain ssz information in the field tags of the data structures. For example, go doesn’t have a way to express an artificial maximum size limit for a given slice so we have a `ssz-max` struct tag to indicate the maximum size considered for ssz encoding. Additionally, we needed a way to represent a fixed size array without needlessly allocating N elements. In other words, we needed a slice that was of length `N` for encoding purposes but wasn’t actually allocating `N` memory with empty objects. For this, we have defined the struct tag `ssz-size`.

Image for post
Image for post
Example fields from the BeaconState with a fixed ssz size for mainnet configuration.

So we solved that problem, but what about handling different configurations? Now it gets a bit tricky. We’ve hard coded these values at compile time and can’t change them at runtime. This leaves us with two immediate solutions. 1) Redefine each data structure for each configuration. 2) Abstract the structs in with interfaces. The first option is a miserable experience since go does not allow for inheritance or generic structures. The second option is more promising, but interface abstractions tend to lead to more cumbersome code and every value read is a copy of the data so it becomes more expensive on the garbage collector and memory usage.

Neither of those solutions left us feeling very great. So we thought about it a bit more. Since we don’t really need to change the configuration at runtime, we started to think about build time configuration. Using our build tool, Bazel, we were able to invoke compile time template substitution so that we could toggle configuration of these hard coded value without any additional code changes.

Image for post
Image for post
Example fields from the BeaconState with template values.

The templated variables randao_mixes.size and active_index_roots.size would be substituted and compiled with the appropriate configuration value, be it mainnet or minimal configuration for SSZ encoding. Read more in 3062.

Sparse Merkle Tree Optimizations

One of our bottlenecks that did not manifest in our old testnet but did as soon as we attempted a v0.8.1 testnet run was the creation of our sparse Merkle trie used for deposits and deposit proofs within Prysm, as this was actually taking up a large amount of memory and time once we had a large enough number of validator deposits. We took a quick revamp of the package, made it massively simplified, and benchmarked to ensure it has longevity throughout our project.

Image for post
Image for post

Eth1 Mock Server

One of the current focal points of Eth2 implementations is to have some easy, standardized way to mock the “genesis” event of the beacon chain. Many teams are attempting a few different approaches, such as using a yaml file to encode a list of deposits, parse that file, and kickstart their local beacon chain. When approaching this problem, we wanted to ensure our “genesis” event would match the real, production scenario as much as possible. That is, we didn’t want to mock genesis from some file but instead mock it from some set of ETH1 RPC endpoint responses that look like the real thing, with people depositing 32ETH into some contract.

Our current testnet does use a real ETH1 RPC endpoint from the Goerli testnet, with a real deposit contract upon which users can deposit ETH to become validators. Ideally, a solution would manifest in the form of some server endpoint Prysm can point to which can give us mock data that would be enough to start the chain.

We created eth1-mock-rpc for this purpose and it gets the job done by reading a list of validator private keys from a local keystore and serving an RPC websocket and HTTP server. This tool will save us hours and hours of development time, being able to kickstart an almost “real” beacon chain from deposit events received via RPC. We thank the awesome go-ethereum project for providing great utilities for interacting with an Ethereum web3 endpoint in Go itself.

Image for post
Image for post
eth1-mock-server responding queries to beacon node

Upcoming Work

SSZ Optimizations

Turns out we weren’t maximizing the usage of our SSZ tree cache for increased performance in our runtime! We spent some time optimizing the cache, making sure it is collision resistant, and overall improving the speed and memory footprint of the go-ssz project itself. Our focus of the project was to achieve correctness first, and optimization second. Given Go does not have support for generics as it currently stands, we needed to make several compromises and find an approach that would make our project easy to test with a standardized API that matches other serialization libraries. There is a lot more work to do on this front and we expect our SSZ project to be highly optimized by our next update.

Database Refactor

One of our goals this quarter is to refactor our db implementation. Currently we use bolt as our underlying key value store. The primary goals of this refactor is to be able to support a larger variety of backends for our db, re-organize packages in the db so that we have proper separation of concerns. Currently we have all the functionality stuffed into a single package, which makes it difficult to read the code from time to time. Also another important goal would be to optimize how we use bolt to persist all the different structures. Currently with how certain methods are structured, those methods take longer to persist. With this refactor any prysm node should be able to run with any backend ex: A key-value store, relational db,etc. This is tracked in this issue and the design is detailed in this document.


ETH2 Development Experience Presentation in BuildEth

Our own Terence presented in BuildEth last friday on the topic of ETH2 client development experience. The talk consists of Prysm client architecture, what it takes to translate spec code to production code and how to manage a good open source project implemented from spec. He highlighted the importance of understanding the spec before coding up the spec and it’s best to leverage open source infrastructure out there to facilitate the implementations. The recording has not been posted but you can check out the slides here.

Ethereum 2.0 Implementers Call

We have just completed our bi-weekly ethereum 2.0 implementers call. As research winds down on the beacon chain, majority of the focuses now revolves around researching phase 1, phase 2 and networking challenges. We are excited for the next milestones on multi client interoperability and contribute to research challenges such as light client syncing and execution environment prototyping.

Interested in Contributing?

Image for post
Image for post

We are always looking for devs interested in helping us out. If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).

Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 0 milestone along with a specific project it belongs to.

As always, follow us on Twitter or join our Discord server and let us know what you want to help with.

Official, Prysmatic Labs Ether Donation Address


Official, Prysmatic Labs ENS Name


Prysmatic Labs

Implementing Ethereum 2.0 - Full Proof of Stake + Sharding

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store