Writing Accurate Time-Dependent Truffle Tests

Paul Razvan Berg
Aug 11 · 4 min read

When you’re building a dapp for continuous salaries, writing accurate time-dependent tests is a necessity. I recently learned how to do it the hard way, bashing my head against the wall, so I wrote this article to spare you the hassle.

I will further assume that:

  1. Your testing framework is Truffle, Ganache + Mocha.
  2. You want to your tests to be as accurate as possible (<5 seconds error margin).

Manipulating time in tests written for Ethereum smart contracts comes with some gotchas.

Gotcha #1: RPC

While it is totally possible to jump forwards and backwards in time in Ganache, there are multiple ways to achieve this. To get accurate results, you must use the evm_mine RPC method in the following way:

Also as a promise in javascript:

Notes:

  1. Without the NUMBER_OF_SECONDS parameter, the RPC call increases only the block height but doesn’t jump in time.
  2. The “id” parameter is optional but good to have. It doesn’t really matter what value you put in there while testing.

There is also evm_increaseTime, which increases the “internal clock” of Ganache so that whenever the next block is mined it has a timestamp offset. This adds overhead though:

That’s right, you’d have to make two RPC calls, compared to only one with the first approach.

Props to Jakub Wojciechowski for coming up with PR #13 for ganache-core. It would’ve been hard to write accurate tests without a deterministic and atomic way to jump in time in Ganache.

Gotcha #2: Run Time

Code itself takes time to execute. Specifically, javascript promises take a non-negligible amount of time to resolve.

This is obvious, right? When writing time-dependent tests for Ethereum smart contracts, things get delicate.

Consider the following:

  1. The time it takes for the test case to run
  2. The amount of seconds you want to jump in the future
  3. The unix timestamp for the moment when you submit yarn run test in your console

Contingent on these variables, your test block may get caught in between the passage of one or more seconds, hence your assertions may break.

For instance:

What this does is that it calls the Sablier contract to withdraw a previously deposited wad of money. The rules that dictate how much the caller can withdraw are specified in ERC-1620.

I expected the test to pass consistently, but I was wrong. So wrong. It started to break ~1 in 8 times and the thing I feared most happened, that is, non-deterministic variance.

I logged the unix timestamp in mocha’s before block and I measured how long it takes for the test to run using node’s performance timing api:

If you add 115 to 1565455128964, you end up with a number that ends with 9079, thus the number of seconds increases from 8 to 9. This is what broke the assertion, because I was expecting a balance of x, when I was actually getting x + 1 (more seconds passed = more money).

While it’s impossible to write a program P1 that can compute how long it takes for another program P2 to finish (see Turing’s Halting Problem), we could safely assume that no more than 1 second should pass between your beforeEach and it blocks. This is assuming the back and forth communication between your node instance and ganache is almost instantaneous, even when running coverage.

Here’s the fix:

Where ONE_UNIT is one monetary unit allocated per second, as per the Sablier model. It is imperfect, but way better than using a “greater than” or “less than” equality check.

Finally, as the OpenZeppelin team argues here, you might not need this level of precision. If your dapp doesn’t involve timestamps or block numbers directly, tolerating larger chronological offsets is perfectly fine. Nonetheless, it’s good to be aware of run time.

Gotcha #3: BeforeEach and AfterEach

Your mileage may vary, but you may want to jump forwards in “beforeEach” and jump backwards in “afterEach”. This is because your contract might have some variables defined in the scope of the “describe” block and you want to run a sequential set of “it” blocks that all assume the same state. Not reverting back in “afterEach” would only increase the timestamp forever.

Example:

As you can see in the snippet above, we have three tests in which we assume the amount withdrawn is 5. In the context of Sablier, advancing in time 15 seconds would yield a withdrawable amount of 15, thus we have to go back to the original state in the “afterEach” block.

Gotcha #4: Snapshots

While truffle provides its own clean-room environment, it’s not a bad idea to implement your own snapshotting mechanism. That is, going back to the original state after all tests are done. It may be helpful for CI or other external environments.

Define those functions in your codebase and then insert his in one of your root test files:

Voilà, now your blockchain will revert to its original timestamp after all magical time jumpings.

Wrap Up

Some of the bits and pieces used throughout this article are inspired or taken from other writings, such as Ethan Wessel’s amazing Standing the Time of Test with Truffle. ̶T̶h̶e̶ ̶o̶n̶l̶y̶ ̶c̶a̶v̶e̶a̶t̶ ̶w̶i̶t̶h̶ ̶t̶h̶a̶t̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶i̶s̶ ̶t̶h̶e̶ ̶u̶s̶a̶g̶e̶ ̶o̶f̶ ̶b̶o̶t̶h̶ ̶e̶v̶m̶_̶m̶i̶n̶e̶ ̶a̶n̶d̶ ̶e̶v̶m̶_̶i̶n̶c̶r̶e̶a̶s̶e̶T̶i̶m̶e̶,̶ ̶a̶n̶d̶ ̶w̶e̶ ̶e̶x̶p̶l̶a̶i̶n̶e̶d̶ ̶a̶b̶o̶v̶e̶ ̶w̶h̶y̶ ̶t̶h̶i̶s̶ ̶i̶s̶ ̶n̶o̶t̶ ̶i̶d̶e̶a̶l̶.̶

Update: Ethan was really cool and he updated his post and ganache-time-traveler package with a few methods that use evm_mine as indicated in his article.

Also, here’s a very good StackExchange thread on the inherent security of block.timestamp and some GitHub threads that shed some light on the history of deterministic time jumping in Ganache (1 and 2).

Thanks for reading! More on Sablier:

If you want to get in touch personally, I’m on Twitter and Keybase.

Sablier

The protocol for real-time finance on Ethereum

Paul Razvan Berg

Written by

Founder @SablierHQ

Sablier

Sablier

The protocol for real-time finance on Ethereum

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade