Running Truffle Unit Tests with Jest
Follow us on a journey and explore how unit tests in Truffle are executed and how we use the Jest test framework to improve interoperability with IDE’s like JetBrain WebStorm and Visual Studio Code
The Truffle framework comes with a handy cli tool that has many useful commands to compile, test, and deploy your smart contracts.
truffle test for example, runs all your unit tests and uses the Mocha test framework to execute them. Many IDE’s like JetBrain WebStorm and Visual Studio Code have advanced features to support running Mocha tests from within your IDE including better navigation in test results, running/debugging specific tests, and reexecuting tests after file changes.
IDE’s usually support all those features through the Test framework’s test runner and their corresponding command line options. The problem now is that truffle test does not use the default mocha test runner (mocha cli) and does not support the standard command line options but instead implements a custom Mocha test runner with their own command line options.
Stefan van den Oord has written a nice article on how truffle test can be integrated into IntelliJ IDEA, so that at least some of the features like the test result explorer work.
Another feature of IDE’s is that they display badges next to your test functions which you can use to run/debug your tests. IDE’s look for test framework functions that represent test cases (e.g. describe, context, and it) and display the badges accordingly. Truffle introduces their custom contract function which wraps the standard describe function and adds default before, beforeEach and afterEach lifecycle hooks. The result is that the IDE won’t recognize your tests anymore because it now sees the contract function instead of the describe function and won’t display the corresponding badges anymore.
Truffle test internals
The tuffle test command does actually much more than just running your unit tests with Mocha. Before running the tests it
- starts and connect to ganache or to your configured network if available;
- prefetches all available accounts from the connected node;
- compiles all solidity files (includes unit tests written in Solidity);
- deploys all artifacts to the connected node;
- provides globals for easy access within your tests (artifacts, web3, assert, expect, contract).
As said before, the contract function adds default before, beforeEach and afterEach lifecycle hooks. The before hook will reset all smart contracts by either using Ganache’s snapshot feature or by redeploying all smart contracts. The beforeEach and afterEach hooks are used to print out the event log in case a test failed.
After all unit tests ran, truffle test disconnects from the node.
Using the standard Mocha CLI
Instead of using truffle test, we could try to solve all features with the standard Mocha framework and use the regular mocha cli to execute our unit tests. To initialize everything we have to run asychronous logic before executing the unit tests. Mocha supports two methods to achieve this:
- pass the -delay option. Mocha won’t execute your unit tests before your code calls the injected run function.
- use a global before function and return a promise which resolves ones your initialization is done.
The problem with both options is that Mocha will require your test files even before the initialization is complete — only the execution of tests and lifecycle hooks is delayed. It would not be possible to access artifacts outside of these functions. This would be very inconvenient and would be a common source of errors.
So, by default, Mocha does not provide a propper solution to this initialization problem. There exists an issue to that problem. Also there exists mocha-prepare which works a bit like the truffle test command — do the initialization first and then start Mocha with the little advantage that it would still support all standard mocha command line options.
Using Jest test framework
As it turns out, the Jest test framework has much better support for early initialization than Mocha has. You can specify a globalSetup and a globalTeardown function through the Jest configuration. Jest will only start once the promise returned on the globalSetup function is resolved.
Furthermore you specify a custom testEnvironment class which can specify setup / teardown function for each test file. This allows us to reset the smart contract once per test file. This differs to the truffle test behaviour as truffle testresets smart contracts always before executing tests within the contract function and not once per test file.
On the other hand, Jest also has some shortcomings:
- By default, Jest executes tests in parallel. This will do more bad than good in truffle projects as we’re connected to a single node. We can run all test linearly by providing the -runInBand command line option or by providing our own test runner through the configuration file.
- Unlike Mocha’s afterEach lifecycle hook, it is not possible in Jest to figure out the test result of the executed test and therefore Jest can’t dynamically print the event log whenever a test failed.
- You can’t pass additional command line options to the Jest cli. Additional options such as the network to use must be passed through environment variables
truffle-jest npm package
All in all we felt more comfortable with Jest, which is why we created the truffle-jest npm package. It extracts the functionality of the truffle testcommand into a new package and adapts it to the Jest test framework.
To use Jest in your truffle project run
npm install — save-dev jest @types/jest truffle-jest
and create a corresponding jest.config.js and jest.setup.js in your project. For more details on how to setup truffle-jest, take a look at the README.
Note: Tests have to be independent of each other if you want to execute single unit tests from within your IDE or from the command line. You can add a beforeEach function to your jest.setup.js that resets the smart contracts before every test instead of before every test file (see the screenshot above).
You should now be able to configure Jest in your IDE or to use it from the command line.
Currently, truffle-jest only supports tests written in JavaScript. Tests written in Solidity are not supported yet. Expect truffle-jest to have some rough edges, as this is a very new npm package. We’ll keep improving it over the next few week.
Conclusion
Truffle comes with awesome support for Ethereum smart contract development out of the box.
By using only the standard cli and test functions of a test framework like Jest, we can archive better IDE integration and can benefit from standard command line options, for example, to only run specific tests or watch for file changes.
Please give truffle-jest a try if you like it and let us know what you think. Also, we will be attending the ETHBerlin hackathon and would love to talk with you there!
We appreciate your support and are always open to any ideas or suggestions that you may like to make regarding of Asure project.
Have any questions?
Have any questions?
Join our Telegram: https://t.me/AsureNetwork
Follow us on twitter: https://twitter.com/AsureNetwork
Check out our website: https://www.asure.network
If you liked the information, clap to encourage us to do more.
