Web3.js v1.0.0-beta.38

The goal of this Medium post is to announce the new version 1.0.0-beta.38, to give closer insights into the development process and to explain the current structure of the Web3.js team.

For the last few years, the former maintainer, Fabian, did maintain and design the API of the Web3.js project. He also was the initiator of the EIP-1193 (EthereumProvider) and created the first draft of it. Fabian is now working on his own project called LUKSO and moved out as the maintainer of the Web3.js project.

Thanks, Fabian for working on and maintaining the Web3.js library for the last few years!

Since Fabian gave over to focus on his new project, I’ve taken over his role as maintainer of the Web3.js library. During the first months, I’ve noticed there would be enough work for one additional developer. That’s why the Web3.js project will hire a second developer for having a higher output.
Further information about the additional developer will be published in a separate announcement later.


Deprecation of 0.20.x:

The version 0.20.x is in maintenance mode since 19.04.2018 and just got security updates and smaller bug fixes. Because of the coming stable release of the library, I’ve deprecated version 0.20.x and recommend switching to version 1.0.0-x of the library if you have not already. Further details of the roadmap of the library are explained below.


EthereumProvider (EIP-1193):

As you may have heard or read, the Provider Ring has finalized EIP-1193. This EIP specifies a provider interface for interacting with an Ethereum node.

The current release of the EIP-1193 is set around April 1st.
Until the first of April, Metamask, Mist and other applications release the EthereumProvider in multiple stages. The last release stage will be to remove the old provider API and to provide a “window.ethereum” property which contains the EthereumProvider. The Web3.js library already has implemented all of these stages and is also compatible with the old providers which are injected from applications like Status, Mist or Metamask.


ENS (EIP-1577):

The ENS team did create the EIP-1577. This EIP defines the “contenthash” field of an ENS resolver. This field got introduced because several applications like Metamask or Status started to resolve ENS names to content hosted or distributed systems such as IPS or Swarm. For further information about the EIP-1577 please go to the specification here and also check out the ENS module documentation of the Web3.js project for the new methods we provide.


TypeScript:

First of all, thanks to @joshstevens19 and @huanzhang89 for creating and testing the TypeScript type definitions! 🙏

During the refactoring and the implementation of the new provider API, started a discussion about the type definitions of the Web3.js library. The topic of it was if we should maintain the definitions in the DefinitelyTyped repository or if they should be placed directly in the repository of the Web3.js.

For supporting the discussion about where they are placed, I created a poll on Twitter to let the broader community decide where the definitions should live. I saw this as a test if the community would like to vote on usability questions over twitter.

It’s no longer required to install the types with “npm install @types/web3.js” because the type definitions are now placed in the Web3.js NPM package.

dtslint:
There was just one point which could have had an impact on the quality of the type definitions. The argument was the tool from DefinitelyTyped which provides testing possibilities for type definitions. Fortunately, Microsoft did release last year the type definition testing tool dtslint which we are using now to provide the same quality of types we could provide over DefinitelyTyped.

Modules with type definitions:

  • web3
  • web3-bzz
  • web3-core
  • web3-core-helpers
  • web3-core-method
  • web3-eth
  • web3-eth-abi
  • web3-eth-accounts
  • web3-eth-contract
  • web3-eth-ens
  • web3-eth-iban
  • web3-eth-personal
  • web3-net
  • web3-providers
  • web3-shh
  • web3-utils

Please open a pull request or an issue in the GitHub repository of the Web3.js project if there is a wrong or missing type definition.


Issues:

During the refactoring of the library in the last few months could I fix a lot of issues. I’ve started to test all of them which are labeled with the label “bug” and closed them or asked further questions but didn’t go through all of them until now. If you would like to contribute and accelerate this process, I would very much appreciate help testing whether the remaining open bug issues are still relevant in the latest release.

The beta.38 release contains fix for over 30 bug fixes. Please see the most recent closed issues in the release notes for details.


Documentation:

Because of the bounty I’ve created together with Gitcoin, Web3.js got way more pull requests in the last few months regarding documentation fixes than usual.

We were able to merge 19 pull request with fixes for the documentation! Thanks all for your support! 🙏

web3-examples:
Because of the bounty for creating live examples did I create the repository web3-examples. The target of this repository is to have starter projects, live examples for the documentation and examples of custom JSON-RPC methods or Web3.js modules. (Bounty | Repository)


Bundles:

With the refactoring, I’ve removed the Gulp build pipeline and installed the Rollup.js bundler which is to prefer if someone is building a library such as Web3.js. I’ve configured the rollup bundler for creating commonJS, UMD and ESM bundles and all dependencies will be handled as externals from Rollup.js. This means that the newly published NPM package will just contain the Web3.js code and nothing more. Webpack or other bundlers will handle all the external dependencies and bundle them correctly.

The full bundle size of a plain Web3.js with Webpack decreased from 6.2Mb to 1.0Mb. This is still really huge for a plain Webpack project but it is a step in the right direction. I will improve the bundle size later more with the simplification of the folder structure back to a similar kind of structure we know from the v0.20.x. But the real reason for the huge full bundle are the packages which Web3.js depends on.

This is why I would recommend you to just install the required modules of Web3.js

But the new folder structure will improve the way how you use single modules of Web3.js. This because we will then export all modules over one index.js, which gives you the possibility to just import the modules you would like to use. Please have a look at the “Namespace” draft below if you’re interested to be involved in the discussion of the new way Web3.js will expose the API.


Quality Assurance:

Unit Tests:
During the refactoring, I rewrote all the tests of the Web3.js library to mocked unit tests, because I do not really like Mocha for testing javascript code. Did I decide to use Jest, because it includes a lot of tools which other “test suites” do not provide by default.

coveralls coverage comment

With the refactoring, I was able to increase the test coverage by 9.6% from 83.4% to 93%. I also moved the tests from the global tests folder to the modules the tests belong to. For each class and file exist a test, with the same name suffixed with “*Test” placed in the module folder of the tested class.

ESLint & Prettier:
I’ve installed and configured the tools ESLint and Prettier for checking and improving the code style. With the combination of ESLint and Prettier does it not only lint the code it will also fix the occurred errors. This will help us to keep the code style quality of the library as high as possible.


New option properties for Web3 modules:

I’ve added during the refactoring of the core module and the creation of the AbstractWeb3Module some additional option properties for customizing the transaction confirmation workflow.

These are the currently available option properties on a Web3 module:

  • defaultAccount 
    The default account that will be used for call requests and transactions.
  • defaultBlock
    The default block which will be used for call requests.
  • defaultGas
    The default gas limit which will be used for a transaction.
  • defaultGasPrice
    The default gas price which will be used for a transaction.
  • transactionBlockTimeout 
    This can be used with a socket provider and defines the number of blocks until the PromiEvent rejects with a timeout error.
  • transactionConfirmationBlocks
    This defines the number of blocks it requires until a transaction will be handled as confirmed. The PromiEvent will resolve with the desired receipt when enough confirmations happened.
  • transactionPollingTimeout
    This defines the polling cycles amount when you send a transaction with the HttpProvider. The PromiEvent rejects with a timeout error when the timeout got exceeded. (1 cycle == 1sec.).

Web3.js — Module API

The provided method extend does no longer exist. This because of architectural reasons and also because I think the preferred way to extend or customize the Web3.js library should be to create a Web3 module.

Methods:
During the refactoring of the core-method module did I create method objects for all JSON-RPC methods the Web3.js library does provide. I also created the AbstractMethod, AbstractCallMethod and AbstractSendMethod. These abstract method classes give us the possibility to quickly implement new methods, to fix a category of methods with just smaller changes in one or two files and the abstraction gives us the possibility to extend from existing methods. The ContractSendMethod, for example, is extending from the SendTransactionMethod and will be used to execute a SmartContract method. This means it’s no longer needed to inject a custom provider for implementing a custom JSON-RPC method now you can just extend from the method you would like to customize.

I will create a section in the documentation with a tutorial, which will explain how a developer could create his own custom method in depth later.

If you would like to know more about then feel free to have a look at this code example.

Subscriptions:
The subscriptions are created the same way as the methods.

Modules:
Before the refactoring, the web3 module got extended with the call of a “core.packageInit()” method in the constructor. Because of the implementation of an AbstractWeb3Module is it now possible to simply extend from the ES6 class for getting all the default functionalities of a Web3 module.

Because you can extend from the AbstractWeb3Module is it way simpler now to create your own custom Web3 module! 
You could create for example custom methods, subscriptions and contract objects and put them together as a Web3.js module for providing a javascript contract API of your project.

I’ve created a small module with a custom “eth_call” method here. I will create later a tutorial section in the documentation about how to create a Web3.js module.

Providers:
I’ve cleaned up the providers and added the long-missing feature of reconnecting on connection troubles with a
WebsocketProvider. Currently, only the WebsocketProvider does support reconnecting and resubscribing. The reconnecting feature will be added to all other socket based providers after we tested and improved it with the WebsocketProvider.

I’ve removed the web3-providers-* modules and created a web3-providers module as a replacement for them with all the providers and related classes. The interface of the internal providers did slightly change. The send methods are now compatible with the EIP-1193. Be sure you checked the provider you are using in the module for getting further information about the provider interface changes. (web3-providers types | implemented providers)

All socket based providers do emit the following events:

  • ready
  • connect
  • error
  • close
  • Also events with the payload id or subscriptions id as the name will be emitted from each provider when the node is responding.

The EthereumProvider and the MetamaskInpageProvider do additional emit the following two events from the EIP-1193:

  • networkChanged
  • accountsChanged

Web3.js after 1.0 the LTS (stable) release:

In my spare time, I got some ideas of how the future of the Web3.js project could look like. I wrote down some of these ideas in the following gist files:

The goal of them is to involve the community in the process of planning the future API and modules the Web3.js library should have. Hopefully, many of you will be a part of this discussion and help me to improve the Web3.js library in a way you want it.


Next Steps:

The last few months, version 1.0 didn’t get monthly releases with bug fixes and new features and I apologize for this. But the clean up of the code base and the improvement of the quality assurance did have a higher priority to me.

The release cycle for version 1.0 of Web3.js will go back to a monthly release cycle and I will have a focus on bugs, stability and the bundle size. It will also be the goal to publish each month another Medium post to get some insights into the ongoing development process and the latest release.

If you would like to support the Web3.js project then feel free to have a look at these Gitcoin bounties or at our issue list on GitHub.

The goal is to release a stable version of the Web3.js library in the first half of 2019. These are the things that have to be done to call the version 1.0 stable:

  • Fixing of most known issues
  • Integration tests
  • Optimizing the dependency graph and simplifying the folder structure.
  • Solidity revert method handling
  • ENS support for input’s of type address
  • Adding tutorials and live examples to the documentation

Thanks all for supporting and helping me to improve the Web3.js library.

Best regards,
Samuel Furter

Github: nivida
Twitter:
Samuel Furter