Writing Accurate Time-Dependent Truffle Tests

Paul Razvan Berg
Aug 11, 2019 · 4 min read
Image for post
Image for post

When you’re building a dapp for continuous salaries, writing accurate time-dependent tests is a necessity. I recently learned how to do it the hard way, bashing my head against the wall, so I wrote this article to spare you the hassle.

I will further assume that:

  1. Your testing framework is Truffle, Ganache + Mocha.
  2. You want to your tests to be as accurate as possible (<5 seconds error margin).

Manipulating time in tests written for Ethereum smart contracts comes with some gotchas.

Gotcha #1: RPC

While it is totally possible to jump forwards and backwards in time in Ganache, there are multiple ways to achieve this. However, to get accurate results, you should use the evm_mine RPC method in the following way:

As a promise in JavaScript:

Notes:

  1. Without the NUMBER_OF_SECONDS parameter, the RPC call increases only the block height but doesn’t jump in time.
  2. The “id” parameter is optional but good to have. It doesn’t really matter what value you put in there while testing.

There is also evm_increaseTime, which increases the “internal clock” of Ganache so that whenever the next block is mined it has a timestamp offset. This adds overhead though:

That’s right, you’d have to make two RPC calls, compared to only one with the first approach.

Props to Jakub Wojciechowski for coming up with PR #13 for ganache-core. It would’ve been hard to write accurate tests without a deterministic and atomic way to jump in time in Ganache.

Gotcha #2: Run Time

Code itself takes time to execute. Specifically, javascript promises take a non-negligible amount of time to resolve.

This is obvious, right? When writing time-dependent tests for Ethereum smart contracts, things get delicate.

Consider the following:

  1. The time it takes for the test case to run
  2. The amount of seconds you want to jump in the future
  3. The unix timestamp for the moment when you submit yarn run test in your console

Contingent on these variables, your test block may get caught in between the passage of one or more seconds, hence your assertions may break.

For instance:

What this does is that it calls the Sablier contract to withdraw a previously deposited wad of money. The rules that dictate how much the caller can withdraw are specified in ERC-1620.

I expected the test to pass consistently, but I was wrong. So wrong. It started to break ~1 in 8 times and the thing I feared most happened, that is, non-deterministic variance.

I logged the unix timestamp in mocha’s before block and I measured how long it takes for the test to run using node’s performance timing api:

If you add 115 to 1565455128964, you end up with a number that ends with 9079, thus the number of seconds increases from 8 to 9. This is what broke the assertion, because I was expecting a balance of x, when I was actually getting x + 1 (more seconds passed = more money).

While it’s impossible to write a program P1 that can compute how long it takes for another program P2 to finish (see Turing’s Halting Problem), we could safely assume that no more than 1 second should pass between your beforeEach and it blocks. This is assuming the back and forth communication between your node instance and ganache is almost instantaneous, even when running coverage.

Here’s the fix:

Where ONE_UNIT is one monetary unit allocated per second, as per the Sablier model. It is imperfect, but way better than using a “greater than” or “less than” equality check.

Finally, as the OpenZeppelin team argues here, you might not need this level of precision. If your dapp doesn’t involve timestamps or block numbers directly, tolerating larger chronological offsets is perfectly fine. Nonetheless, it’s good to be aware of run time.

Gotcha #3: BeforeEach and AfterEach

Your mileage may vary, but you may want to jump forwards in “beforeEach” and jump backwards in “afterEach”. This is because your contract might have some variables defined in the scope of the “describe” block and you want to run a sequential set of “it” blocks that all assume the same state. Not reverting back in “afterEach” would only increase the timestamp forever.

Example:

As you can see in the snippet above, we have three tests in which we assume the amount withdrawn is 5. In the context of Sablier, advancing in time 15 seconds would yield a withdrawable amount of 15, thus we have to go back to the original state in the “afterEach” block.

Gotcha #4: Snapshots

While truffle provides its own clean-room environment, it’s not a bad idea to implement your own snapshotting mechanism. That is, going back to the original state after all tests are done. It may be helpful in CI or other external environments.

Define those functions in your codebase and then insert this in one of your root test files:

Voilà, now your blockchain will revert to its original timestamp after all magical time jumpings.

Wrap Up

Some of the bits and pieces used throughout this article are inspired or taken from other writings, such as Ethan Wessel’s amazing Standing the Time of Test with Truffle. ̶T̶h̶e̶ ̶o̶n̶l̶y̶ ̶c̶a̶v̶e̶a̶t̶ ̶w̶i̶t̶h̶ ̶t̶h̶a̶t̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶i̶s̶ ̶t̶h̶e̶ ̶u̶s̶a̶g̶e̶ ̶o̶f̶ ̶b̶o̶t̶h̶ ̶e̶v̶m̶_̶m̶i̶n̶e̶ ̶a̶n̶d̶ ̶e̶v̶m̶_̶i̶n̶c̶r̶e̶a̶s̶e̶T̶i̶m̶e̶,̶ ̶a̶n̶d̶ ̶w̶e̶ ̶e̶x̶p̶l̶a̶i̶n̶e̶d̶ ̶a̶b̶o̶v̶e̶ ̶w̶h̶y̶ ̶t̶h̶i̶s̶ ̶i̶s̶ ̶n̶o̶t̶ ̶i̶d̶e̶a̶l̶.̶

Update: Ethan was really cool and he updated his post and ganache-time-traveler package with a few methods that use evm_mine as indicated in this article.

Also, here’s a very good StackExchange thread on the inherent security of block.timestamp and some GitHub threads that shed some light on the history of deterministic time jumping in Ganache (1 and 2).

Thanks for reading! More on Sablier:

If you want to get in touch personally, I’m on Twitter and Keybase.

Sablier

The protocol for real-time finance on Ethereum

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store