By “process” here I mean something that produces the changes of rules in Ethereum (someone would say “protocol changes”, but I am going to try to avoid the word “protocol” here, because it would make things very ambiguous from the start). Some examples of changes of rules:
- New EVM opcodes
- Changes in the gas schedule (operations require more gas than before, the reverse has not happened yet, but it might happen in the future)
- Making certain things invalid (like deploying contracts with bytecode larger than 24 kB)
- Changes in what certain pieces of data mean, or what certain operations do.
In the illustrations below, I wanted to explain things without always going down to all the details, so I most definitely omitted some important teams and people, and some steps in the process I either was not aware of, or chose not to show. Please let me know in responses if you think I missed something significant and thank you in advance. Please also feel free to modify this and publish your own versions of the process!
What I think the process has been so far
People on the internet have ideas about how to change (improve or not) Ethereum. They write EIPs (Ethereum Improvement Proposals). For most ideas, this is where it ends.
Some EIPs attract attention of one of the three client implementation teams, and get picked for prototyping and further evaluation. I put those three implementation teams in the diagram, because each of them is “special” in its own, important way:
- Parity — second most used implementation (in terms of number of number of nodes in the network), but apparently most used among mining pools.
- Geth (go-ethereum) — most used implementation (in terms of number of nodes in the network).
- Aleth (original C++ implementation) — the only implementation currently capable of “generating” the consensus tests.
The short conclusion is — if nobody in these three teams wants to (and has time to) prototype the change, it is unlikely to happen.
The time period between Byzantium (October 2017) and Constantinople (February 2019) required a lot of work on the client implementations to make sure their performance can keep up with the increased transactional load and growing state. That might be one of the explanations why Constantinople does not really have any significant changes that required non-trivial research and/or implementation (for example, account abstraction and extended block hashes).
Prototype implementations usually lead to the EIPs being refined, more correctly specified, or sometimes thrown out as flawed in some way. Also, prototype implementation makes it easier to create test vectors — description of various scenarios under which the change of rules is effected.
Test vectors need to be written down in a special format, sometimes called “filler tests”. Here is an example (one of the CREATE2 tests).
I said before that Aleth (C++) implementation is special because at this time, the tool for converting those filler tests into the consensus tests (this tool is called “testeth”) is tightly coupled with Aleth. This effectively makes implementations of EIPs in Aleth reference implementations. Consensus tests, generated in such ways, can be run against most of other implementations.
Testing team is also special, because this team creates most of the test vectors (EIP writers usually don’t add a lot).
I will not draw more pictures here, but the process does not end there. Client implementations use consensus tests to find bugs (in their implementations, in the reference implementation, and in the specification). Consensus tests are also used to drive testing in the “Hive”. There is also fuzz-testing involved. (I am not familiar with the setup though). Then there are also test nets (and that requires another entire post of its own).
What the process could be in Ethereum 1x
Now, instead of waiting for some developers in well-established implementation teams to get interested in a proposal, it is possible to form a working group and start working on it. Of course, it is still not trivial, because it requires finding people who want to do the work, people who want to pay for it, etc. But at least it hopefully gives us a nice separation of duties, and a more scalable process.
Working group picks a client implementation that its members are most familiar with, and uses it to produce a reference implementation for the change. Out of the reference implementation, it then produces EIP (which is already much higher quality than in the current process), and test vectors. This is already possible today.
Something that does not yet work today is the generation of the consensus tests out of a non-Aleth reference implementation. But we hope to develop the integration of the new tool “retesteth” with the most popular implementations . Initially, I thought about creating a working group for that, but then realised it might actually be quicker just to do it. Here is some of the on-going work on it (we also making fixes in retesteth).
Find people and Funding
How do we find people to work in the working groups? How do we make sure the work is funded? These questions are not trivial, but not impossible to answer. So far I see two main challenges to overcome:
- The pool of people we draw from is quite limited. Perhaps this is because “Core Developement” is still seen like some kind of Dark Arts. The solutions I see are in better self-description and education. More on this will come later (but not in this post)
- Going from funding a few implementation teams continuously and letting them do “their stuff” to funding more specific and temporary initiatives requires looking at funding through different lenses. How much “due diligence” and oversight is too much (in terms of overhead), who can decide whether working groups actually deliver, etc. This is also solvable, and also more on this will come later (not in this post).
What is the process for Ethereum 2.0?
I suspect one of the reasons for the monumental pivot that has occured in June 2018 from the “evolution” path (PoW->hybrid PoW/PoS->PoS->sharding) to the “complete rewrite” path was the realisation that Ethereum change process is way too slow to carry Casper and Sharding ahead with the necessary speed
You may know that such pivots happen all the time in lots of systems. Usually these pivots are motivated by the frustration with the challenging aspects of the legacy system (like existing customers), or with the incumbent engineers who maintain the legacy system and won’t allow changing it quickly enough. In my opinion (and from my limited experience), such pivots are only really sucessful if at least one (or better two) following conditions are met:
1) The rewrite is performed by a more able and experienced team than the one maintaining the legacy system
2) The development and change process has been dramatically changed to address the most pressing challenges of the “legacy” process
The way I see it, Ethereum 2.0 has not yet landed to the point where the development and change process kicks in, but we do not know what this process will look like. So I assume people would try to fall back on the same process that exists in Ethereum 1.0. But… it is that process that the Ethereum 2.0 pivot sought to side-step.
What I am really saying is that there is an under-appreciation of how important it is to invest into improving the development and change process in Ethereum 1.0 to make sure Ethereum 2.0 lands on a good ground when it gets released. None of this will be a problem, of course, if Ethereum 2.0 will be perfect from the start and won’t need changing :)
This was somewhat rushed description of what I am having in mind when I say that Ethereum 1x is not just a set of roadmap items, but also an attempt to create a new, more efficient and inclusive process of delivering improvements to Ethereum.