My first months as the maintainer of Web3.js

On the first of July, I started as the new maintainer of the Web3.js library. In the beginning, it was really challenging for me, to get an overview of the library I should maintain. Not because I didn’t understand how to interact with the JSON-RPC API of Geth and Parity but mostly because of the code complexity I’ve encountered. This didn’t hold me back from implementing the new ABICoder of ethers.js, the ENS module and to fix some issues. Finally on the 27. August I managed to get my first release published:

Release

The feedback I got from the community, because of the support of the new ABI Library, was just great and I felt really energized. This was exactly what I needed to get into the working mode I would like to be.


Issues:

I’ve spent two working days after the first release just to go through all the issues Web3.js had on GitHub. It was sometimes really hard to determine if these issues are because of Web3.js, of a misconfigured environment or if they were because the developer is using the API wrong. But it was only hard to determine because of the lack of information on some issues, that’s why I’ve created an issue template and also a pull request template. I think many of you were wondering why I assigned a lot of them to me, this was just because I needed something like a checklist to know which issues I’ve already gone through and which I don’t. After the cleanup of the issue list, I was fixing the “low hanging fruits” just to send a signal to the community that Web3.js is back! I know there are still a lot of open issues on GitHub but believe me, I’m working as hard as I can to solve as many of them as possible and to bring Web3.js 1.0 to a level of stability the community would like to have it.

EthereumProvider and the related refactoring:

After the cleanup of the issue list, I’ve started to implement the EthereumProvider (EIP-1193). There were two ways to implement the new provider spec. First I could implement it in a hacky way using some ifs all over the Web3.js library or I could do it the clean way with some adapters and a resolver to have internally always the same interface.

I decided to take the clean path for implementing the EthereumProvider it was clear to me that the implementation of the adapter pattern and of a resolver is not the fastest way but I thought it’s time to stop hacking and start engineering.

Because of the new way I handle the providers internally I was able to remove the web3-core-requestmanager module from the Web3.js library. We all know that classes and modules that have words like “manager” in his name are a sign of bad code architecture:

“ Classes and objects should have noun or noun phrase names like Customer, WikiPage, Account, and AddressParser. Avoid words like Manager, Processor, Data, or Info in the name of a class. A class name should not be a verb.” 
- “The Clean Code” — Robert C. Martin

Because of the removal of the web3-core-requestmanager I had to refactor the core, Eth, Bzz, Shh and Net modules because all of them were related to the RequestManager.

Core:

The web3-core module had a object with the following methods:

packageInit & addProviders:

Both methods extended the behavior of the class on construction. Anyone who knows how to build a SOLID and clean code base should know that changing the behavior of a class on construction is an anti-pattern and should not be done. The correct way to share logic between classes is with inheritance.

Both of them were related to the RequestManager and because of this, I had to refactor them. Thats why I decided to remove them and create an AbstractWeb3Module. I think some of you would now say inheritance is not possible with ES5 but definitely, it is.

Example:

/**
* The abstract class
*
* @param {String} name
*
* @constructor
*/
function Abstract(name) {
this.name = name;
}
/**
* Logs the name property of this class
*
* @method someMethod
*/
Abstract.prototype.someMethod = function () {
console.log(this.name);
};
/**
* The concrete class
*
* @param {String} name
*
* @constructor
*/
function Concrete(name) {
Abstract.call(this, name);
}
// Inheritance from Abstract class
Concrete.prototype = Object.create(Abstract.prototype);
Concrete.prototype.constructor = Concrete;

I totally agree with you this does not look like clean code but it’s definitely better than to have something like a packageInit method in a constructor. The same code in ES6 does look way better:

/**
* The abstract class
*/
class Abstract {
/**
* The abstract class
*
* @param {String} name
*
* @constructor
*/
constructor(name) {
this.name = name;
}
   /**
* Logs the name property of this class
*
* @method someMethod
*/
someMethod() {
console.log(this.name);
};
}
/**
* The concrete class
*/
class Concrete extends Abstract {
/**
* @param {String} name
*
* @constructor
*/
constructor(name) {
super(name);
}
}

As you can see it’s possible to write JavaScript in a clean manner. If you would like to see the AbstractWeb3Module please have a look at it here. I will also write a contribution tutorial for developers who would like to add a Web3Module to the library.

The AbstractWeb3Module is not as good as it could be but I will improve it in another refactoring.

I’ve also refactored the way how Web3.js adds JSON-RPC methods to a Web3Module. Before the refactoring, methods have been added on construction with a forEach loop over an array. I’ve told you before it’s not recommended to extend or change the behavior of a class on construction that’s why I decided to create a Proxy which delegates the calls. This was in my point of view the one and only clean solution to provide the same public API as before in a clean manner.

Core-Method:

The core method module was one very complex file and I took me a long time to create a model of it because of the complexity I’ve encountered in this file.

Some key points to it:

  • 600+ lines
  • Nested methods on three levels with recursion.
  • It took me around one day to write down a model of the transaction confirmation workflow.
  • Bindings

Yes, it’s true I could just ignore this code but I couldn’t sleep with the knowledge that there is such a file in my library where I should have to be proud of. Because of this, I decided to refactor it from one really complex file to this:

Some key points to the new structure:

  • Separation of concerns
  • Open close principal
  • Dependency injection
  • Dependency inversion
  • Factory pattern
  • JSON-RPC methods implemented as MethodModels (each method is just implemented once)

Because of the separation of concerns, it’s way easier to understand what exactly is going to happen on a method call. Another advantage we have because of the dependency injection is, that we can finally write fully mocked unit tests. I’ve created a module factory for the dependency resolution, this way we can easily create classes with his dependencies just with calling one method and we have just to pass external (from another module) dependencies or parameters over the method arguments as you can see here.

Each module does now contain an index.js, this file is the public API of the module and the only entry point to classes of a module.

The same type of refactoring was done for all other core modules. I can recommend, to have a look at the ethereumProvider branch, if you would like to see more JavaScript magic and please feel free to give some feedback!

Eth: Before & After
Bzz:
Before & After
Shh:
Before & After
Net:
Before & After

Unit tests:

I’ve moved from mocha to jest just because I will have a test suite and not just a test runner. Additionally, jest provides a really cool code coverage tool, is sandboxed by default and is ready for further refactorings (TypeScript).

ES6:

As you saw from the URLs above I’ve ported the whole library to ES6.

Bundles:

As many of you noticed on bundling of the Web3.js library the bundle size of the library was around 6Mb. That’s why I’ve removed the legacy gulp bundling and moved to rollup which all big projects are using (React, Angular, Apolo etc.). Because of the bundling with rollup, I can now finally bundle a commonJS, UMD and ESM bundle! After the release of this refactoring, NPM will no longer contain all files of the Web3.js library. It will just have these three bundles who is also another improvement because we all know how big the node_modules folder anyways is.

Integration tests:

Yes, I’ll create integration tests for the JSON-RPC API! This will increase the stability of the library because I’ll get “notified” if the Geth, Parity or Ganache API has changed and I can finally guarantee that all provided JSON-RPC methods will work.

Roadmap for the stable 1.0 release of Web3:

  • Finish the EthereumProvider implementation and the related refactoring.
  • Supporting of hardware wallets (Trezor, Ledger)
  • Full support of ENS (It should be possible to use ENS names for all from, to and address arguments of a method)
  • Improvement of the reconnect handling with a socket connection and with subscriptions.
  • Fixing of all critical issues.

Conclusion:

  • Never felt this kind of energized for a project as now!
  • I really like to be a part of the decentralizing movement and I see all of the opportunities right before me.
  • I never thought I could manage to get from legacy code to maintainable code in just ~2 months.
  • There is still a lot to do after the refactoring and also after the stable release.
  • I would like to implement way more Web3Modules to support anything a developer needs to build a Dapp and especially I would like to write some tutorials and best practice for the development of a Dapp.
  • It’s definitely possible to write well organized and SOLID JavaScript. I hope many other developers will adapt and improve the solutions I’ve found!

Hopefully, I can start with this medium post an open discussion about:

“How to build a smart contract API?”

Best regards,
Samuel — Web3.js Lead Developer