Prysmatic Labs Ethereum Development Update

Onwards to “The Merge”

It’s been a while since we’ve done one of these updates, as our team has been working towards our biggest milestone, “The Merge” of Ethereum proof-of-stake into the current Ethereum network. There is a lot happening on this front, with a lot of public communications handled by Tim Beiko who coordinates work between teams like us and teams working on current Ethereum clients such as go-ethereum.

In terms of complexity of code in Prysm, we foresee the merge to be even simpler than the Altair hard fork in terms of its changelog. Instead, a lot of complexity lies in testing and having the confidence to ship everything out. On our side, some of the major work left has to do with supporting latest testnet initiatives such as Kintsugi, helping on refining the engine API used for “The Merge”, and designing a plan to handle edge cases in chain synchronization. An example of ongoing work is the design of Optimistic Sync. We will have a lot more updates on the merge as a result of the latest testnets in January.

Major Improvements

We have a ton of major improvements and features to our software we are proud to share with you, including some that are in the pipeline and will make a big difference to running a validator with Prysm.

New Features

Tracking Individual Validator Indices

Our teammate Potuz has put a lot of work into a feature allowing for new metrics for individual validators in our Prysm beacon nodes. There is a lot that happens during a node’s runtime, including a lot of state transitions, p2p networking, handling of blocks, attestations, and other messages. For operators running a lot of validators, it is useful to understand not only how their validators’ balances are changing, but also how they are performing in the context of the blockchain’s consensus protocol. For example, we should be able to track details about their attestation’s inclusion, attestations and blocks in the p2p network for specific validator indices, and more.

By adding the flags:

beacon-chain --monitor-indices=1 --monitor-indices=2

…your beacon node will expose metrics specific to those indices as it tracks their performance at runtime. This feature was suggested and initially implemented by the Sigma Prime team, the folks behind the Lighthouse Ethereum consensus client. This feature will be available in the next Prysm release, v2.0.5, which will be the last planned one for 2021, with documentation available at the time of release.

Checkpoint Sync

We have been working on implementing checkpoint sync in Prysm. This feature will give users the ability to sync a beacon node from a checkpoint, known formally as a Weak Subjectivity Checkpoint, rather than processing the entire chain history from genesis. That means faster initial sync times and better security against long range attacks when a source is used to obtain the checkpoint block and state. This will make the experience of running a Prysm node so much easier, and allow everyone to sync quickly with the head of the chain. The safety of checkpoint sync is a really interesting line of research, as it offers strong mathematical confidence. Vitalik explains this idea elegantly in his blog post “How I Learned to Love Weak Subjectivity”.

P2P & Core Optimizations

A new feature that has been created in Prysm is batched p2p gossip message verification. Currently, Prysm verifies each message separately from gossip networking, however a few months back a feature was released which allowed nodes to perform batch signature verification of gossip messages. This feature leads to significant reduction in CPU usage.

Potuz has also benchmarked a much faster hashing algorithm for SHA256 that is 10 times faster than current Prysm and other consensus clients. This is game changer as hashing is one of the most common operations done in a node at runtime, and the new approach is under development.

Moreover, a much more efficient HashTreeRoot computation for the beacon state is being introduced. The Altair hard fork introduced some changes which leads to state root computation being more expensive. A new feature has been released that reduces this bottleneck and allows nodes to have a more efficient solution.

Database Optimizations

Zahoor from our team has been working on foundational database optimizations. We’ve been doing a deep dive into how data is stored in Prysm, and have found many places that should be revamped. Our philosophy is that we should never store the same data more than once on disk, and currently, there are many large data structures that are being repeated, leading to large DB sizes. Every time we save a beacon state, it has a large validator registry in it. Instead, we save the registry separately and only store a changelog for each validator. This feature requires a database migration, but in the future, we working towards a more maintainable strategy for users to leverage. If you desire, you can migrate to this more efficient schema using the --enable-historical-state-representation flag, which could take almost an hour to complete and there is no going back unless you re-sync a node. For now, we only recommend this to folks that don't mind that one-time-migration downtime.

Standards and Usability

In terms of standards we are supporting, we have collaborated with the other consensus client teams to create a standard Keymanager API which is used for doing validator key management over REST HTTP. This is the first step towards creating multi-client point-and-click interfaces to import and migrate validator keys! In the next Prysm release, we will have full support for this standard.

Additionally, we have been aiming to improve user experience of the Prysm web UI, and instead of asking for so many passwords, we instead use a token-based approach for authentication, thanks to James from our team This is now available in our latest Prysm release. You can read the docs on how this works here. We have been putting our more regular updates to the web UI, including nicer error handling, smoother features, and a much more advanced onboarding architecture.

Testing

Testing is critical for consensus, especially with so much on the line. Since the beginning, our team has put a lot of work into creating an end-to-end test suite that runs as closely as possible to the real thing. Our goal was to launch an actual beacon node, validator, and eth1 node from a deposit contract to kick-off a chain. Now, every pull request to Prysm runs our end-to-end suite, which gives us a lot of confidence in ensuring the chain runs as expected. In each run, we trigger a lot of different evaluators that compare different parts of the runtime including balances, finality, slasher, chain sync, and more.

As a next step, we plan on improving our native end-to-end suite by adding more eth2 clients into the mix. This multi-client E2E is expected to allow for more extensive testing and increased confidence in any new commits merged to our default branch. The ultimate end plan is to have all the clients running in an automated fashion with a simple command. This will allow us to test new forks, features with increased confidence. To begin, we are adding the Lighthouse client into the mix.

New Teammates

We have grown our team by quite a few folks over the last few months as we get closer to the merge. Some of the individuals that have joined us are

• James He, as a full-stack engineer from J.P.Morgan Chase & co. with experience in UI/UX. James has been putting a lot of work into helping Prysm adopt the standards around APIs, web interfaces, and key management
• Artem Vorobiev, as a security engineer with many years of red hat hacking experience, helping us understand and find all kinds of exploits at every level of development
• Taranpreet Singh, as an infrastructure engineer working to get our infrastructure, CI/CD as solid as possible on the way to the merge and beyond. Taran has years of experience in our infrastructure stack
• Potuz, a protocol researcher that has contributed significantly to the Ethereum consensus specs and to important decisions in the Prysm software as a community member, has now joined our core team

Interested in Contributing?

We are always looking for devs interested in helping us out. If you know Go or Angular and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you.

Check out our contributing guidelines and our open projects on Github.

As always, follow us on Twitter or join our Discord server and let us know what you want to help with.

Prysmatic Labs Ether Donation Address

0x9B984D5a03980D8dc0a24506c968465424c81DbE

Prysmatic Labs ENS Name

prysmatic.eth