Nomic Labs DevX: 1 year working on Ethereum developer experience
The very beginning of our developer experience collaboration with the Ethereum Foundation dates back to July 2019, and since it’s been just over a year since we started the formal engagement it’s a good time to recapitulate and reflect upon what progress has been made.
This is the third post in my DevX series outlining the work we’ve been doing around developer experience. In the first post, I explain from a very high-level perspective what kind of things we’ve been doing, and in the second one, I enumerate what our core areas of focus to prioritize projects have been. In this third one, I’m going to discuss each specific project we’ve worked on, and share links to our research and related Github links.
Overall we’re pretty satisfied with the results since many of these have been very meaningful contributions to Ethereum developer experience and productivity. This isn’t meant to be a humblebrag, but we initially foolishly believed we could achieve more.
We underestimated two things:
- How long it would take to build a holistic end-to-end perspective on Ethereum as a software development platform and the needs of developers, to be able to prioritize effectively.
- How long it would take to get other Ethereum core libraries and tools to integrate our proposals and pull requests. Understandably these teams and projects all have their existing roadmaps and priorities, but nonetheless, we heavily underestimated this factor.
This list mainly represents the quick fixes and low hanging fruit we identified across the platform with the potential to deliver large returns relative to their cost. We’re now working on a completely different kind of roadmap with projects with much longer time horizons, which we’ll share in time.
Now, without further ado, this is everything we’ve worked on so far in the 2019–2020 Nomic Labs DevX season, categorized by the same product drivers defined in the second post, in no specific order:
Speed: Make the contract development feedback loop faster
1. Solidity compiler macOS native binary
From our experience building Hardhat we knew that the macOS native binaries weren’t being distributed, and this was the main reason for the widespread usage of the much slower JavaScript binary. We ran benchmarks to check speed differences across the different kinds of binaries and confirmed that the native binary offered 5–20x compilation time improvements relative to the JS binaries. We presented our research to the Solidity team, provided assistance in producing the builds for macOS, and let the other tooling teams know about this. Now the default for tools can change from JS to native compilation and deliver very significant speed improvements across the board.
Resources:
- Nomic Labs research document
- Github issues #3168 #9258
2. JavaScript EVM optimization
From our experience building Hardhat Network and its stack traces, we knew that there were significant optimization opportunities to make the EVM implementation in EthereumJS run faster. This is the EVM that is used in Hardhat, Remix and Ganache, so this was highly promising as a way to have an impact across the ecosystem. We hired EthWorks to run benchmarks on the library and they detected a 100x performance regression and several substantial optimization points. This project led to the EF JS team making performance a new global objective, as well as incorporating many of the findings and the benchmark itself into the library’s CI.
Resources:
- EthWorks raw result
- Nomic Labs research report
- Github issues #775 #865 #64
3. Hardhat compilation pipeline optimization
Hardhat’s compilation speeds were relatively fast already, even though this area hadn’t been heavily optimized, so there were still significant potential gains on the table. With Hardhat’s growth this year, this couldn’t be ignored anymore as a source of meaningful impact to developer experience. We optimized it by switching from compiling the entire Solidity codebase every time to only including the dependency graph for the source files that were modified since the previous compilation.
Resources:
- Hardhat announcement blog post, with some numbers
- Github pull request
Visibility: Increase visibility to what’s happening at the EVM/Solidity level
4. Solidity debugging symbols
Part of the reason why Solidity debuggers struggle at being sufficiently reliable has been insufficient debugging metadata produced by the Solidity compiler. We did a deep dive into Solidity step debugging, brought together all ecosystem players for whom this is relevant and collected their perspectives and feedback to identify and finally design the missing metadata necessary for effectively implementing a step debugger.
Resources:
- Our proposal
- Github issues #9462 #9461 #9332
5. Solidity stack traces
So much has happened since 2019 that it feels like a very long time ago, especially given that in the past year Ethereum developer experience has seen radical improvements, but back in 2019 working with Solidity was still a black box effort. There was very little information to work with when code failed. Implementing stack traces was our first substantial contribution to Ethereum developer experience, and it was the birth of Buidler EVM, which today we know as Hardhat Network.
Resources:
- Announcement blog post
- Devcon 5 talk
- Hardhat Network stack traces documentation
- Ecosystem reaction on Twitter
6. Solidity console.log
Once we had nailed our EVM execution inspection code infrastructure to generate stack traces, we went after the next big missing essential in Ethereum: console.log. This remains our project with the biggest impact ROI, and the spec implementation was executed by the Tenderly team.
Resources:
7. Solidity clear and explicit error messages
When we launched Buidler EVMHardhat Network, we combined stack traces with the detection of Solidity/EVM failure cases. Major props to the developers who managed to debug their code before this, and our condolences for the number of hours it required. Using many low-level EVM heuristics we detect at runtime the reason why a transaction is reverting and elucidate the back-then typical “Transaction reverted” into one of:
- Calling a non-payable function with ETH
- Sending ETH to a contract without a payable fallback or receive function
- Calling a non-existent function when there’s no fallback function
- Calling a function with incorrect parameters
- Calling an external function that doesn’t return the right amount of data
- Calling an external function on a non-contract account
- Failing to execute an external call because of its parameters (e.g. trying to send too much ETH)
- Calling a library without
DELEGATECALL - Incorrectly calling a precompiled contract
- Trying to deploy a contract that exceeds the bytecode size limit imposed by EIP-170
Low-key this has been the essential cherry-on-top of the stack traces + console.log + error messages Solidity debugging productivity trifecta.
Resources:
Editing: Improve Solidity editing assistance across major editors
8. Solidity editing assistance
With a specific set of tools configured in the right way, Solidity editing can be less than terrible, but there’s a long way to go when it comes to enhancing a developer’s ability to write Solidity code. We have been working for a few months on a new Solidity editing experience, which we’ll launch once it’s polished enough. It’s going to be the most advanced Solidity editor out there, with type-smart autocompletion and more. We intend to 80/20 the initial implementation and over time start building the different primitives needed for a proper implementation across popular editors. Watch out for the announcement!
Tooling: Lower the barrier to entry for tooling development
9. Export Solidity’s compiler parser
The ability to parse a programming language is a core pillar of a tooling ecosystem developing, and Solidity parsing today remains an unsolved problem. There’s an ANTLR-based parser with many forks, which is useful, but it doesn’t completely address the set of needs. Since there isn’t an official parser library, nor an official grammar definition, the fastest shortcut to having a reliable parser that always works down to the latest and smallest detail, is for the compiler to externally expose its internal parser. This will enable richer tooling to exist. We went deep into the compiler and prototyped it to understand the feasibility, and then presented it to the Solidity team. This has been implemented, and we’re now hoping to get it backported to Solidity 4, 5, and 6 to have full language coverage.
Resources:
Composability: Reduce friction when integrating protocols
10. Sophisticated deployment framework
There’s been some ecosystem progress in deployment tools, but we still haven’t seen a 10x hit. We’ve been working for a few months on a sophisticated framework based on dependency injection that we think will step up the game significantly, and enable valuable things for developers to more effectively test their protocol integrations moving forward. This research project is being led by Tenderly, and we’ll publish and announce once the MVP is ready.
Other
11. Web3.js
If you’re a web3.js user, you probably know that the library has radically improved over the course of the last year. We brought attention to the urgent need for this to happen to EF management in mid-2019, and then got intimately involved with the project as hands-on advisors to help stabilize it. We proposed a path forward, reviewed pull requests, pushed for stability-oriented processes and policies to be established, and helped grow the team behind it. In the end, after the project was stable, this led to the maintenance being outsourced to ChainSafe, who have a team better suited for the task.
Resources:
- Github issue #3070
12. Github Solidity support
Full support for Solidity on Github has a long history with 17 previous attempts over the course of years. Up until this project, a dot file in the repository opting into enabling Solidity highlighting was the best that could be done, but Github was still not detecting Solidity as a language used in the repositories, and beginners hardly knew this Github trick. The challenge was deeply rooted in Linguist, Github’s language detection library, so we reached out to the project maintainers and offered to sponsor a contribution to perform the refactoring needed. A full-time week of work later in addition to some coordination with Github staff, and we got official support deployed.
Resources:
- Github issue #4936
- Ecosystem reaction on Twitter
13. Removing node-gyp dependencies across the ecosystem: ethereum-cryptography
This is one of the projects we’re the proudest of because it was personally something we really despised and knew was really affecting the entire ecosystem. There are a lot of details in the blog post linked below, but the summary is that we replaced the cryptography dependencies used across the Ethereum stack (15+ pull requests across 10+ different components) to achieve significant installation time optimizations (5–10x scale) and significantly improve reliability across almost every JS component. We contracted Trail of Bits to audit our code and collaborated with Tenderly and many other teams to implement it across the ecosystem.
Resources:
14. EIP1193
This EIP is about the interface for the object dapp browsers amd wallets like MetaMask expose to interact with Ethereum. This interface wasn’t standardized before and had subtle discrepancies both in the projects exposing it and in the projects consuming it, leading to a messy situation with unreliable compatibility across wallets and libraries. Developers needed to know the right component versions that were compatible with each other, and users were experiencing obscure issues depending on which wallets they were using. We reviewed all past discussions to collect all perspectives and find an approach that satisfied most stakeholders. We organized a meeting about this at Devcon and coordinated the discussion online as much as we could. Erik Marks from MetaMask and Ryan Ghods from the Ethereum Foundation were the ones that ended up taking it to the finish line.
Resources:
- Github issue #2319
15. solidity-coverage stabilization
Back in 2019, this project had gotten into a tricky situation. There were a series of bugs that users had written bug-fixing forks for, and these forks started being used across the ecosystem. The fork that each person was meant to use depended on which bugs that person’s tests triggered, and there were some additional reliability issues. Code coverage is an essential tool to build safe contracts, so we gave Chris Gewecke a grant to focus full time on the project for a bit and we collaborated with him to design a revamped architecture that would be more reliable.
Resources:
16. JSON-RPC API standardization
The API Ethereum nodes and development networks expose to communicate with them isn’t standardized, and there are different versions of it across the different components. This is an issue, but the bigger issue is that even the parts that are consistent across implementations present immediate opportunities for improvement that will make interfacing with them easier. To enable an orderly process to iterate this interface at some point, we kicked off a standardization effort for this API. Before we decided on this, there was an existing intention within the EF that had been stalled for a while for this to happen under the guidance umbrella of the Oasis standards body. We reignited this effort and gave a grant to Nethermind for Tomasz Stanczak to coordinate the process, who’s made meaningful progress behind the scenes.
Resources:
Closing thoughts
These are the projects that are finished and ongoing, but there’s more in the backlog to produce improvements across all product drivers, especially in editing and tooling.
Overall it’s been an awesome year full of rewarding work and meaningful impact, and we couldn’t be any more grateful for the love the ecosystem has expressed towards us and our work 💞.
