The state of client diversity in Ethereum
A research report based on the Ethereum critical installation survey
A couple of months back, Ethereum core developers found themselves at a crossroad when the main agenda item for biweekly AllCoreDevs meeting was Features vs. Maintenance on the Ethereum mainnet. As Berlin, the next network upgrade was being planned, the pressure to add the new functionality to clients and deploy it on testnets was increasing. At the same time, the infrastructure to ensure that client implementations were properly tested was being maintained by a small subset of core developers and Geth’s usage dominance on the network meant that team felt the most pressure because they would bear most of the responsibility if anything went wrong.
Over the next few calls, what the root cause of this problem was and how it could potentially be solved was discussed extensively, leading to a survey analyzing the various critical users of the Ethereum network. Here is a timeline of events as well as a summary of the survey’s findings.
Timeline
Jun 25, 2020
The discussion about the toll that features were taking on core developers was initiated in the Core Dev Gitter where Alexey Akhunov shared his thoughts on another discussion related to ReGenesis. He said, “I would love us to have a pause and talk about issues of burn-out, complexity, implementation diversity, features vs tech improvements. ReGenesis is not a silver bullet, but it is an attempt to be constructive.”
Jun 26, 2020
In the subsequent All Core Dev meeting, Alexey proposed a “bargain”, whereby Ethereum devs should set some performance and diversity goals and agree that new features will only be added once these goals are met. This meant to agree for a complete pause on the readiness of the next network upgrade — Berlin.
The question that the group was trying to find an answer was:
“How do we balance features vs. keeping the network healthy, avoid burning out implementers, promote implementation diversity, etc.?”
Ethereum developers are very well aware of the value of diversity. Martin S. said, “Back in 2016, diversity was good. We want to have diversity if there’s an issue with Geth, Ethereum doesn’t go down.”
Greg Colvin mentioned, “To some extent, Geth having the big majority is inevitable…Unless we have a plug and play standard, we can’t take Geth out and put something else in if there’s a problem.”
Hudson Jameson said that he’s on the fence about the freeze, but that some things can be done to improve client diversity. For example, to reach out to miners, to ask what it would take for them to run non-Geth clients.
Alexey added a comment about making it easier for non-core devs to contribute because if there are more client teams to reach out to, it will make it easier for people to engage.
In response to how client diversity is managed in Eth2 and his thoughts on the topic, Vitalik said, “One thing we did is not create a situation where one client had a head start. We’re not launching Phase 0 unless we have multiple clients.”
As it turned out, there was enough food for thoughts to continue the discussion in the next meeting.
July 02, 2020
Applying the “Five Why’s” to the Client Diversity Problem
Piper Merriam, who couldn’t join the ACD meeting#90, shared his thoughts on a blogpost trying to understand the root cause, to avoid wasting effort focusing on the wrong problems. Summary of the problem statements are:
Problem Statement #1: Geth represents a super-majority of the Ethereum mainnet and is maintained by a very dedicated, but small team. The current conditions under which the client is developed are not sustainable and lead to client burn out.
Problem Statement #2: The Ethereum mainnet does not have adequate client diversity. Because of this, a critical consensus bug, or another denial of service attack affecting the Geth client could severely impact the smooth operation of the network.
Problem Statement #3: Creating an Ethereum client is too difficult. This has lead to poor client diversity, which leaves the network in a vulnerable position.
Problem Statement #4: The current structure of the Ethereum network results in it being prohibitively difficult to create a client for the network. This results in the network having very few clients, and thus, poor client diversity.
Piper mentioned that staying away from the “solution” was intentional, however, if the problem with client diversity is accepted, then he believes there is a strong case to be made that can be fixed by focusing on the network and protocols themselves.
July 10, 2020
Continuing the discussion on network health, people agreed that the group probably should spend some time dissecting the problem.
Problems identified
- Ethereum protocol is inherently monolithic and therefore very hard to implement. Do people need to spend time to structure the protocol and make it more modular?
- What makes a person a superhero who can know everything about what you need to do? Is it possible to avoid that up to a certain extent?
- Understanding the rationale for having several clients, does every client need to reimplement every single piece of code
- There should be a way for the new clients to be able to ditch the old EVM rule and just start with the rules in the network that exist today.
Suggested solution
- Stateless Ethereum is a good starting point, it will probably address a part of the problem. Also, Alexey’s “three networks” model likely solves the “networking protocol being monolith” issue.
- Generic reusable components — Focus on simplification of reusable client components but to get there clients need to be more modular so that you aren’t required to implement all of the things.
- Team structuring — some level of structuring might help to improve the life of the developers.
- The ReGenesis suggestion by Alexey would also lower the barrier for new client developers.
July 22, 2020
CoreDevCalls 91 Retrospective: How did we do with “Five Why”s?
Goals
In this retrospective, Alexey highlighted the connection between two actual goals and the “client diversity”.
- Resilience of the network to implementation bugs
- Resilience of the network to capture
He further stated that probably we do not have a good definition of “client diversity”.
“What matters is not exactly the number of installations, but the number of critical installations, which includes mining pools, exchanges with high volume, wallet/Dapp backends, Etherscan.”
And came up with a suggestion that the Ethereum Cat Herders may create a reliable system that would give info about the critical installations, but without revealing too much info about them. So that important metrics can be tracked rather than vanity metrics.
July 23, 2020
The Ethereum Cat Herders created a Critical Installations survey and shared it with the community.
Survey results
The survey result suggests that the client adoption ratio does not seem to be ideal for the diversity of clients but not bad either. One key point to mention here is the charts below are based on the participating institutions and not the actual nodes on the network. Hence, we see an overlap in the percentage distribution.
Q. Client installed and Client’s version (eg. Geth v1.9.17)
This chart acknowledges the dominance of Geth followed by Parity-Ethereum and OpenEthereum. Based on data collected, Besu, Nethermind, Trinity, Aleth, and TurboGeth do not seem to be a choice for critical installation like mining pools, exchanges, wallets, etc. But again, the survey sample was small.
Q. The number of nodes installed per client? (eg. Geth — 10, OpenEthereum — 5)
Geth is the most popular among Dapps whereas Parity-Ethereum is the choice for wallet that responded to this survey. Mining pools, exchanges, and other infrastructure providers indicated using a combination of clients by sharing the number of nodes per client.
Q. If you’re willing to consider a backup client installation (an archive node, other than your present client), which would be your preference?)
This question was added to collect the data on the popularity of clients other than Geth, and Parity-Ethereum. People seem to be open to OpenEthereum more than Parity-Ethereum. Besu and Nethermind turn out to be the next popular choice of critical installations.
Participants of the survey
We’ve considered data submitted for critical installations only. Duplicate submission and hobbyist data are not considered for this report. Responses include data from 4 mining pools, 1 wallet, 1 exchange, 2 Dapps, 3 other infrastructure providers.
Q. Any additional information, you would want to provide?
Looking at the participation rate in the initial days of the survey, my thought was maybe people don’t prefer to respond to surveys or generally don’t want to share such information, which stands correct but with email reach, responses increased. Due to the decentralized nature of the Ethereum community, collecting responses form every stakeholder is challenging however our reach indicated that people are happy about this initiative and are willing to open a channel of communication. Some of the responses received to this optional question are:
#client-development channel is created in Eth R&D discord for people to participate in the active discussion
We communicated the issue to the OpenEthereum team and also shared OpenEthereum’s Discord channel to directly communicate with the developers on the request of the client team.
Conclusion
Based on the limited responses that we’ve received, the data for client diversity is not the same as reflected on the Ethernodes but very much provide the same picture of the reality. Metrics collected indicates:
- Geth seems to be the most adopted client followed by Parity-Ethereum and OpenEthereum.
- Mining pools, exchanges, and other infrastructure providers are inclined towards using multiple clients instead of sticking to one.
- Besu and Nethermind are the next popular choice of critical installations.
- With a very small variation on the client version, overall nodes seem to be in the sync with the mainnet.
- A significant percentage of nodes are still running ParityEthereum which is not maintained or upgraded. Instead, Parity gave official control and governance of the Parity client to the newly formed OpenEthereum team.
The Ethereum Cat Herders are committed to providing support to the developers as well as the community out there looking for information on progress and other support.
(Extending thanks to Alexey Akhunov for inviting ECH to support, Tim Beiko for reviewing, fellow Cat Herders and everyone who participated in the survey.)
Join Ethereum Cat Herders discord at https://discord.gg/sgdnxZe
Edit: The cover image of the article is updated on the request of the OpenEthereum.
