Eth 2.0 Dev Update #54 — So Close to Official Multi-client Testnet!
Onyx Testnet Updates
Bad blocks propagated in Onyx
Last week, the Onyx testnet experienced a few hours of volatile peer to peer messaging. Users were reporting high error rates in processing new blocks and it seemed to affect the whole network. This issue caused a period where the beacon chain was unable to finalize. We identified the root cause to be a Prysm client which was out of sync and producing blocks with a parent older than the last finalized block. In the Ethereum 2.0 specification, blocks cannot build upon old finalized blocks. However, the specification allowed for blocks which violated this condition to be “valid” for propagation. The result was that clients would receive the block, forward it through the network, then fail to process it. The issue was first reported on github and was resolved within a few hours. The fix has landed in Prysm alpha.15 release and a recommended specification change has been submitted to the eth2 spec repository.
Prysm accounts revamp underway
For those who have run a validator, you may have noticed that our existing key management tooling isn’t the best user experience, so we’re been working on cleaning it up! A revamp of our account management has been long overdue since the introduction of EIP-2335. With EIP-2335, a standardized keystore directory and file format are standardized for all clients to follow a similar strategy to make swapping between clients seamless! While we’ve been refactoring the key management tooling, we’ve also been making sure our new wallet tooling is much easier to use, including displaying account information in a clean, color-coded form, on top of a streamlined prompt entry for account creation and modifications.
You can track our progress here:
https://github.com/prysmaticlabs/prysm/issues/6327
Merged Code, Pull Requests, and Issues
Big hot/cold state refactoring complete
In the last biweekly update, we mentioned a major refactoring of our state memory management to reduce the frequency of disk operations with regards to saving states. This new feature started with a user report that a Prysm client took several minutes to sync after restarting a fully synced node and ended up with two weeks of design discussion, refactoring, and optimization. We’re really pleased with the initial results (+10% sync speed) and expect this feature to land in the next Prysm release.
Blockchain service refactor and huge bump in test coverage
One of the audit feedback was concerning the around block chain package. The block chain package is one of the critical components of Prysm and it handles beacon block and attestation processing. The package needs to be simple, well understood and with well test coverage. Ever since the feedback, we have been fine combing the block chain package to make it simpler and have fewer interfaces. The summary of the overall work is listed here:
- 6537 — Clean up regular block processing path
- 6562 — Clean up initial sync block processing path
- 6599 — Clean up initial sync batch block processing path
As of overall test coverage improvements, this remains work in progress. We have started narrowing down the testing gaps and improving one file at a time.
- 6613 — Improve test coverage for receive block path
- 6618 — Improve test coverages for logging and tree handler
The overall progress can be tracked via 5783
Block batch verify in initial sync
One of the major improvements required to speed up chain synchronization has to do with how we verify block signatures. The naive way of processing blocks is to perform sequential processing of valid state transitions + valid signatures. Every time we verify an individual signature, we incur a significant performance cost. Instead, we can verify state transitions sequentially and in process, aggregate batches of signatures which we only verify once at certain checkpoints. This can help drastically improve the speed of sync for mainnet. Our team has been working on this critical feature and the improvements are noteworthy. Developer Afri Schoedon, who coordinates the eth2 multiclient testnets, has recently published a report on Prysm’s performance in the Altona testnet published here.
With full signature verification, Prysm is on par with Lighthouse, the rust implementation of eth2. We are confident we will continue improving our benchmarks as we approach a mainnet release.
Remote Signer Reference Implementation for Prysm
As part of our effort to revamp how validator accounts work in Prysm, we have been working on getting great reference examples for anyone running our client to understand how to use our tools. In particular, many users want to run their validator clients in the cloud but want to run a remote signing server elsewhere that is able to securely store keys and sign data as needed. In the Prysm validator client, we initialize a gRPC client that connects securely to a remote signer as desired by the user. For reference purposes, we created an example remote signer server on Github here https://github.com/prysmaticlabs/remote-signer, licensed under Apache 2. You can also write your own remote signer servers in most popular programming languages that support gRPC, as long as your server implements the required API schema for a remote signer defined in Prysm here. We will be adding comprehensive documentation to https://docs.prylabs.network regarding how to use the remote signer and all other accounts v2 related functionality.
Upcoming Work
Rate Limiting and Peer Scoring
Rate limiting and peer scoring refactoring work has been started. Our aim is to make sure that Prysm beacon node is more robust, and is capable of performing its duties in presence of slow, broken, or even malicious peers.
The current rate limiting system is already capable of banning peers that send requests too aggressively, and we are extending it to become more sophisticated in how good/bad peers are detected and what actions are taken to prevent misbehaviour. In addition to banning, we are devising a reputation system that will score peers based on a wide range of different performance metrics (average blocks returned, blocks successfully processed etc), throttling peers that violate carefully selected thresholds. Peers that persistently violate such thresholds will be banned for longer periods of time.
If you are interested in tracking the progress of new peer scoring system please follow the #6622 issue on GitHub.
Interested in Contributing?
We are always looking for devs interested in helping us out. If you know Go or Solidity and want to contribute to the forefront of research on Ethereum, please drop us a line and we’d be more than happy to help onboard you :).
Check out our contributing guidelines and our open projects on Github. Each task and issue is grouped into the Phase 0 milestone along with a specific project it belongs to.
As always, follow us on Twitter or join our Discord server and let us know what you want to help with.
Official, Prysmatic Labs Ether Donation Address
0x9B984D5a03980D8dc0a24506c968465424c81DbE
Official, Prysmatic Labs ENS Name
prysmatic.eth
