I’ve been attending recent Ethereum all-core-dev meetings. These were mainly about the next change in the Ethereum protocol called “Metropolis.” This is my first time participating the preparation of some orderly protocol change. After I had joined in September last year, the Ethereum protocol changed as responses to massive spams, but these changes were specific responses, not a planned milestone like Metropolis.
Ethereum protocol changes are proposed in EIPs: Ethereum Improvement Proposals. Usually, I first see a GitHub issue in the repository, then some discussions on the issue, and finally a pull request in the same repository. The discussion usually continues on the pull request, and controversial points and left-over ambiguities are discussed in the all-core-dev meetings.
Although some of the EIPs are still changing, the set of EIPs for the Metropolis seems stable now. Those EIPs listed under “accepted” section are being implemented for Metropolis. I have been implementing these changes as pull requests to the Yellow Paper (a specification of Ethereum). So, I had to follow the Metropolis EIPs.
EIP is modified
Discussions on the EIP pull requests sometimes lead to changes in the EIP texts. Some discussions continue on the all-core-dev meetings, and after a rough consensus, EIPs change as well. Currently, there are no deadlines for EIP changes for the Metropolis. Whether to change an EIP or not is up to the sense of orderly preparation and the sense of urgency before Metropolis must happen.
One thing I don’t understand is the calculations made in some all-core-dev meetings about the current price of ETH, difficulty and the current block height. These numbers crunched together indicates the point of time when the ice age comes, and the network becomes sluggish. The calculation is supposed to guide the Metropolis preparation. Maybe Metropolis will delay the ice age, but I don’t know for certain. I find it interesting that higher prices allow more time in a quantitative, soft way. If we are too late, people start experiencing annoyances gradually, until the point they will desert the network. Something needs to happen before that. This soft scheduling might behave economically much better than hard deadlines seen in other political or juridical procedures. Just as markets are communication channels, the block time works as a medium between Ethereum users and client implementors. For this soft deadline to be effective, participants need a somewhat accurate idea of what causes what kind of delays (that’s why I’m writing this). And then the economy might work out on its own, hasten and delay preparations, or promote or inhibit EIP modifications. Or that might not be the case, which is why I proposed to freeze EIPs for Metropolis when they are ready, but I feel quite ambivalent now. I don’t know how the organization works because this is my first time. So I watch EIPs and read client codes, and say something when I see something.
Anyway, EIPs change. The last change I noticed was two days ago. So, it’s appropriate to say some Metropolis EIPs are still being amended.
Clients are supposed to implement the changes
When an EIP changes, Ethereum clients are supposed to implement the changes. I don’t know how each client implementor watches the EIP repository. I happen to belong to the C++ team, so sometimes I notice changes and do something, though maybe I’m only the fourth frequent contributor in this corner.
An EIP changes and Ethereum clients implement the change. Also, the client test cases change. I would like to draw attention to this part because these test cases keep the different Ethereum clients together on the same chain.
Test cases are planned
Some people look at the EIPs and fill a spreadsheet “Metropolis tests” with cases to be tested. I tend to look at the EIPs and then look at various implementations, find differences and tricky points, and add these cases to the spreadsheet. Sometimes I simply find the EIP text ambiguous, leave a comment on the EIP, and add that case in the spreadsheet.
Test cases are implemented
Currently, consensus test cases for the Metropolis changes are mainly added as GeneralStateTests. They are called state tests because the test expects certain state root hashes after transactions on a state. They are called general because a single GeneralSateTest case can contain expectations for different versions of the protocol, different values sent, different amounts of gas specified and so on. In general, a general state test describes expected state root for each cell in a multidimensional array of these factors.
We do not calculate the state root hashes by hand. These test cases are produced with the
cpp-ethereum client. The
cpp-ethereum client computes the expected state root hashes in GeneralStateTest cases. When you compile the
cpp-ethereumproject, you get an executable called
testeth. This executable usually tests the
cpp-ethereum client against the consensus tests, but with certain options, the same command can be used to generate the consensus test cases. So,
cpp-ethereum needs to implement the changes before test cases are generated.
testeth command cannot create consensus test cases out of thin air (or it can but only randomly). We need an input file for
testeth. This input file is called a filler, whose name needs a suffix
Filler.jsonin the file system. The filler files are partial test cases with some information omitted. You need to specify the pre-state where the transaction starts, you need to specify what kind of transaction you want the clients to run, but you can specify only some aspects of the post states in a filler file. Sometimes, when I am checking something about a specific corner case, I store one bit in the storage index 0 and I only care about the storage index 0 of a particular account. Then I can only talk about that storage slot in a filler file. I can even specify different expectations on different versions of the protocol, which is useful when I’m writing a test case for a protocol change.
testeth(with a certain option) runs the transaction on different factors, my expectations are met, and produces a proper test case with the state root hash of the post states. The resulting test cases can be used against other clients.
Currently, I spend a couple of days trying these for one EIP. Maybe I become quicker next time, but I doubt it. The process involves jumping from one repository to another, and sometimes even to the third (when a new instruction is needed in LLL). After this, typically we have two pull-requests: one on
cpp-ethereum and the other on the tests repository. The entanglement seems complicated but is manageable. In the past, the process was maybe a little bit simpler because the filler files used to live in the
cpp-ethereum repository, but git clone took so much time that the fillers moved to the tests repository.
I don’t know if we are going ahead or behind the schedule because there are no explicit schedules and this is my first time. Many others look quite relaxed. Maybe they know they can make it with a final rush. I don’t like the final rush, and anyway I need to understand these changes. I need to maintain the EVM for theorem provers. So I’m learning how to add tests, and I’m adding tests for Metropolis changes. Also I need to document how to use
cpp-ethereum to convert fillers into filled test cases. Before that, I wanted to write about the bigger picture. The consensus test writing is certainly a parallelizable realm, so I want the process to be accessible.
Send me a note with your Skype handle if you would like to join the efforts (there is a Skype chat room about this, and I’ll add you there).