DSRV’s Validator Team and Parithosh Jayanthi from the EF DevOps Team discuss their roles, insights, and what it took to prepare for The Merge
In anticipation of the utter landmark that is The Merge, we spoke with both our Validator Team and Parithosh Jayanthi from the Ethereum Foundation’s DevOps Team. Years in the making, Ethereum’s much-awaited transition from Proof-of-Work (PoW) to Proof-of-Stake (PoS) has been a long and meticulous process comprised of many moving parts, stages, and players.
The end goal being a more sustainable and scalable Ethereum, The Merge will essentially fuse the current Ethereum mainnet (to be called the Execution Layer) with the new PoS chain, known as the Beacon Chain (to be called the Consensus Layer).
Part 1 of this interview series will take you through our Validator Team’s pivotal role in The Merge, as well as the lessons learned and stories gained in the process. In Part 2, Parithosh will share his valuable insights to answer any technical questions you may have about merge tests, shadow forks, and clients. Without further ado, let’s get into it!
Part 1: Introducing DSRV’s Validator Team
Q: Can you briefly explain The Merge?
A: The Ethereum network is currently undergoing a transition from one consensus algorithm to another: Proof-of-Work (PoW) to Proof-of-Stake (PoS). While the Ethereum mainnet today is still operating based on PoW, another PoS-based chain known as the Beacon Chain, has been up and running since December 2020. While this chain cannot process smart contracts, it crucially introduces the PoS mechanism via staking ETH.
It’s important to note that the Ethereum Foundation (EF) recently decided to forgo the previously used terms Eth 1.0 and Eth 2.0, in favor of the Execution Layer (EL) and Consensus Layer (CL).
The Merge is essentially an event that will combine the two chains now operating in parallel, changing Ethereum’s consensus algorithm from Pow to PoS in the process. Once complete, Ethereum will be one chain consisting of an Execution Layer (the existing mainnet), and a Consensus Layer (the Beacon Chain).
Q: How has DSRV been participating as a validator on Ethereum?
A: We are presently operating 7391 validator nodes on the Beacon Chain through Lido, an Ethereum Liquid Staking protocol. There are 420,000 Ethereum validators in total, which means that DSRV accounts for more than 1.7% of all Ethereum validators!
Q: What is DSRV’s role in The Merge?
A: Our Validator Team is one of the few external teams to participate in the many tests conducted by the EF in preparation for The Merge. Operating 1.7% of all Ethereum validator nodes, we’ve been diligently working towards a successful Merge from the start.
One way the EF tests The Merge is via Shadow Forks. This effectively simulates The Merge across various networks (mainnet and testnets) in order to spot any remaining issues and evaluate node performance, state syncing and growth. In order to test the Merge within the existing environment without affecting the canonical chain, both the Ethereum mainnet/ testnets and the Beacon Chain are forked.
Ethereum currently has over 10 different client types for the EL and CL. These tests basically allow us to check in advance whether the clients function properly during and after merging. If problems arise, we can analyze the bug and share it with the Beacon Chain Client Development Team.
DSRV has participated in two Goerli testnet shadow forks, as well as six mainnet shadow forks. We were the only validator team to participate in the Mainnet Shadow Fork 11 with the EF, and at the time of writing, are carrying forth the new Mainnet Shadow Fork 13. In addition to the EF and Beacon Chain Client Dev Team, DSRV has also collaborated with Lighthouse and Nimbus on tests. As such, we participated in the Goerli testnet merge, which was successfully carried out on August 11th KST.
Q: What was difficult when participating in the Merge tests?
A: We ran into some difficulties during early shadow forks, like being unable to create blocks or finding bugs in the Ethereum mainnet and Beacon Chain clients. After sharing the presence of these bugs with the Client Development Team, they were promptly fixed; the most recent Mainnet Shadow Fork 12 ran smoothly without any issues.
Q: Any episodes of the collaboration process you wish to share?
A: The first merge test we participated in was the Mainnet Shadow Fork 7, scheduled around 3AM KST. Because the TTD (Terminal Total Difficulty– total amount of mining required to mine the final PoW block) was moved forward suddenly, the merge actually took place early and our Ethereum client failed due to OOM (Out of Memory) 5 minutes before the merge. We tried saving the node while the merge was already in progress, which means we had a hard time synchronizing the already isolated network.
This led us to create a webpage that predicts merge times directly with TTD and monitored nodes, and we haven’t run into this issue since. While August 11th was a celebratory day thanks to a successful Goerli testnet merge, it was also a day of record-breaking downpour and flooding around parts of Seoul (our neighborhood included). So we celebrated a successful testnet merge with rain dripping from the ceiling!
Q: After collaborating with the EF on The Merge, what next?
A: The Merge has been highly anticipated for over two years now, ever since the Beacon Chain first launched in December 2020. Prior to this, DSRV took part in validating the Medalla testnet (final testnet for the Beacon Chain), which launched August 2020. We’re grateful to have been able to contribute our technical expertise as validators, working with a small network of teams towards a truly historic moment for Ethereum. Just as the 12th Mainnet Shadow Fork was successfully completed, we hope to carry through the final Merge with flying colors.
Transitioning to PoS means that the future Ethereum will be secured via staking versus mining, with blocks created through staked ETH. You can currently stake ETH on Ethereum, with a couple setbacks: you must stake a minimum of 32 ETH, and you won’t be able to unstake your ETH before the Shanghai Upgrade, expected in early 2023. Because of this, liquid staking services are emerging in order to provide liquidity for stakers. A prime example is Lido, with other new services like Swell now emerging as well. These allow users to engage in Ethereum staking without needing 32 ETH. DSRV is one of the node operators running the highest number of validators on Lido, and we’ve just started managing a staking pool on the newly launched Swell.
We are also currently working with SSV.Network and Obol to test Distributed Validator Technology (DVT). This technology essentially allows distributed validator nodes to jointly sign messages, improving on both fault tolerance and network security.
It’s been a privilege to take part in such a monumental event, and we hope to continue serving the Ethereum community as best we can after The Merge.
Part 2: Meet Parithosh Jayanthi from the EF DevOps Team
Introductory Questions
Q: Could you briefly introduce yourself and explain which part of EF you’re in charge of?
A: I help handle the DevOps needs for Ethereum upgrades. I helped launch testnets for the Altair upgrade last year and since then have been working on the merge testnets and general testing requirements.
Q: What were you testing by conducting shadow forks and merge tests?
A: In the early days, we used merge testnets to ensure client compatibility. We then migrated to using shadow forks to stress test the clients. This allowed us to observe how the client behaves in a real world environment — with real load and state.
Technical Questions
Q: Can you explain how difficult The Merge is?
A: The Merge is by far the most complicated Ethereum upgrade to date. This is because it introduces a lot of new modular pieces that need to work together. This required us to introduce many new methods of testing to ensure that compatibility works as expected. The fact that we want to upgrade Ethereum without any downtime, makes it significantly more difficult as there are more edge cases.
Q: What did you focus on most while conducting merge tests?
A: A large portion of the testing was focused on ensuring client stability in different scenarios. We had to make sure that we are able to gracefully recover from most, if not all, scenarios that could happen on the mainnet.
Q: What was most difficult while conducting these tests?
A: Coordination of multiple client teams while the specification is actively changing is extremely hard. We were constantly updating either the specs or the clients, making it hard to keep synchronized in such a decentralized environment.
Q: What criteria determines the success/ failure of The Merge?
A: We will be looking at block proposal rates, attestation participation percentage and sync committee aggregate participation rates to determine success/ failure. In the end, if we merge and these values are stable, we’d consider it a smashing success.
Q: If The Merge fails, is there a plan in place such as a rollback?
A: No, once The Merge is done there is no way to reverse it. Any issues found later would be fixed in the merged environment.
Q: Assuming there were some issues during merge tests, could you share which were most memorable?
A: There were definitely a lot of issues during the merged testnets. One such issue happened during the Kintsugi testnet with Marius’ fuzzer producing invalid blocks on purpose. This caused multiple clients to fail, leading to a large portion of the network not finalizing and forking off. We were able to find the root cause and patch the network, bringing it back to stability after almost a week.
Q: There are many different EL and CL clients– Lighthouse, Geth, Prysm, Nimbus, Teku, etc. Why is it important to use this many clients?
A: Ethereum clients are extremely complicated pieces of software. We want to avoid a scenario where a bug in one client is propagated across the whole network. Having multiple client implementations prevents one bug in one client from unilaterally taking down the entire network. Additionally, each client is built with a purpose in mind — some clients are meant for low power devices, some are meant for enterprise users. This means that Ethereum can cater to various demographics at the same time, gaining better security.
Q: Preparing for The Merge has been a lengthy and arduous process, not to mention this event being the first of its kind. Why is all this effort to change consensus algorithms worth it?
A: One of the most underrated advantages of the merge design is the modular nature of the implementation. With this separation of concerns, we will be able to tackle more challenging upgrades that focus on consensus and execution with separate developer teams. This change in consensus algorithm also brings more security to Ethereum at a much lower carbon footprint and cost. The win-win nature of this upgrade makes it worth it.
General Questions
Q: How do you determine when The Merge will take place, and how much time is left until it happens?
A: We rely on Bordel.wtf, maintained by Mario of the EF. His website relies on some statistics to predict when TTD would be hit.
Q: In preparation for The Merge, the difficulty bomb will drop as a means of disincentivizing miners. Can existing miners disable the difficulty bomb and continue mining on the PoW chain regardless?
A: The difficulty bomb was meant to motivate developers to release updates to Ethereum and for miners to regularly keep up. It will be diffused with The Merge. If a miner wants, they are of course free to fork Ethereum and disable it on their fork. But that will bring many other issues to their plate, e.g. maintaining forks of clients, making their chain recognized by the community, etc.
Q: Will the inflation rate of ETH change after The Merge?
A: Yes, the issuance rate of Ethereum will drastically reduce post-Merge.
Q: The existing PoW chain only uses one client, so why does the merged Ethereum require multiple clients?
A: The existing PoW chain would have also benefited from having multiple mining clients, but that didn’t happen for a variety of reasons. However, merged Ethereum will hopefully make Execution Layer diversity a thing.
Additional Questions
Q: You must have collaborated with a number of different teams; how was it to work with DSRV?
A: It was great to collaborate with DSRV! They were extremely proactive in keeping up with the testing timeline and communicative with their findings. We appreciate your help in testing The Merge!
Q: Thanks to you, we were able to smoothly proceed with merge tests, and we also expect the Mainnet Merge to go well. Any last comments before we catch you on the flip side?
A: A huge thank you to all the developers and community members that have worked so hard over the last year to make The Merge possible!
Special thanks to Parithosh for sharing his knowledge and insights with us during an extremely busy time! We hope this article helps readers better understand the colossal significance of The Merge, and the behind-the-scenes work taking place to safely prepare for it.
About DSRV
DSRV is a leading blockchain infrastructure company based in Seoul, South Korea, working with some of the most respected names in the blockchain community to grow the multi-chain ecosystem. Supporting 20+ major protocols, we build entreprise-grade solutions to empower our community to build, stake, and create.
DSRV’s mission is to spearhead the industry’s sustainable growth, and in doing so, provide an access point for all. Whether through our multi-chain node service or our powerful all-in-one staking platform, we exist to push this space forward.
Join our community:
