Back to the future!! Testing Solidity through time with Brownie.
I’ve been having a blast testing some smart contracts I’ve been working on - so much so, I’ve decided to share some of the cool new features discovered in Brownie, including time travel which has blown my mind.
Brownie is a Python-based development and testing framework for smart contracts on Ethereum. → https://eth-brownie.readthedocs.io/
Aside from being an awesome replacement to truffle, it comes baked with some seriously powerful testing libraries - pytest and hypothesis.
Lets do this!
It all starts with one line
brownie init
Few other assumptions for this example: you have a smart contract that you want to test, and it does something over time. It could be accruing interest, vesting tokens over time, FOMO3d etc. In my case, I’ll be going through a crowdsale contract with a start and end date.
We’ll take a mintable token contract. Any will do. Just put a token contract in the /contracts folder, add your crowdsale contract and we’re good to go. I’ll post the repo I’m working on below as a reference implementation, but for now, lets go ahead and compile.
brownie compile
Time to activate pytest
All you have to do is add a .py file in the /tests directory.
That’s it.
If you’re familiar with pytests, you can create a conftest.py file which we’ll keep all our deployed contracts and fixtures. Fixtures are reusable objects we’ll be calling each time we go to the future, and back.
from brownie import accounts
import pytest# Use pytest fixture so we can reuse this
@pytest.fixture(scope='module', autouse=True)
def test_token(Token):
name = 'Test Token'
symbol = 'TEST'
decimals = 18
mintable = True
initial_supply = 0
owner = accounts[0] # Deploy test_token
test_token = Token.deploy(owner
, symbol, name
, decimals
, initial_supply
, mintable
, {'from': accounts[0]}) return test_token
Here we deployed a contract “Token” that is frozen in time, importantly the scope set to ‘module’ allows you to run tests, but after the function finishes, resets the contract state before the next test.
@pytest.fixture(scope='module', autouse=True)
def frames_crowdsale(Crowdsale, test_token):
# Time when you run the test
start_date = rpc.time()
end_date = startDate + 60*60*24
owner = accounts[0] # Deploy crowdsale
crowd_sale = Crowdsale.deploy(test_token
, start_date
, end_date
, {"from": owner})
# Let the crowdsale mint
tx = frame_token.setMinter(
crowd_sale
, {"from": owner}) # Test if successful
assert 'MinterAdded' in tx.events. return crowd_sale
That’s kinda cool, we set up two contracts in Brownie, but when will we see some time travel?!
Well before we get started, we have just activated pytests, deployed two contracts in a fixture, added the crowd sale as the minter and asserted there indeed was an AddedMinter event. We can now run the tests and check the code coverage - which is pretty - with the following lines.
brownie test -v
brownie test -coverage
Setting a start and end time
Previously I would have either test a tiny time window. Or play with Geth and Ganache settings, write out a long perfectly curated script of test transactions to run, avoiding any prior txns errors, restart the node and do it all over again as I develop… I have even heard of someone - deploying to Ropsten, waiting 24 hours and testing again - checking it actually finalized - before deploying to main-net.
“ok it’s been 24 hours lets make sure it finalised”
Still, better than no testing at all - but far from ideal. Instead, let’s do a test in Brownie, going forwards in time, then going back in time & run a second test.
#/tests/test_crowdsale.py# Test 1
def test_finalise(crowd_sale)
with reverts():
# error: Crowdsale not yet finished
tx = crowd_sale.finalise()
# Travel 25hr into the future!
rpc.sleep(60*60*25)
tx = crowd_sale.finalise({'from':owner})
# Check if finished
assert crowd_sale.finalised() == True# Test 2 - Reset back in time
def test_buy_eth(crowd_sale, test_token):
tx = accounts[3].transfer(crowd_sale, "10 ether") # Can still buy tokens!!!
assert 'Purchased' in tx.events
assert test_token.balanceOf( accounts[3]) == 10 * 10 **18
There are in fact a number of ways to use Brownie to skip through time. https://eth-brownie.readthedocs.io/en/stable/core-rpc.htm
After my initial excitement in Gitter, Ben Hauser, the lead developer of Brownie goes to me…
“You should try it in stateful testing”
Stateful testing is a gamechanger! Essentially you set up some rules, set up some truths — called invariants — and let it run until it breaks. You don't have to map out specific user interactions, it tries a number of combinations. It’s fantastic for testing a bunch of edge cases, finding dust, you name it. But it really gets into its own world when you include time travel, as a rule.
https://eth-brownie.readthedocs.io/en/stable/tests-hypothesis-stateful.html
As we approach 1.21 Gigawatts, all you need to gather in the pseudo code below is that there are 4 rules for the contract, and one truth that should hold, called an invariant. The truth being, how much the contract should hold.
#/tests/test_crowdsale.pyimport brownie
from brownie.test import strategyclass StateMachine: eth = strategy('uint256', max_value = "10 ether")
sleep = strategy('uint256', max_value = 60000)
addr = strategy('address') def __init__(cls, accounts, contract):
# Initialise the contract
cls.accounts = accounts
cls.contract = contract def setup(self):
# Reset each run
self.contributed_usd = 0
self.hardcap_usd = 3000000 def rule_1_buyEth(self,eth):
# Purchase in ETH calcs def rule_2_buyUsd(self,usd):
# Purchase in dollars def rule_3_sleep(self,sleep):
rpc.sleep(st_sleep) def rule_4_final(self, addr):
self.contract.finalised() def invariant(self):
# compare the contract amount with the local state
assert self.contract.contributedUsd() == self.contributed_usd
In the reference implementation I go through the logic for each of the rules. I share the rule_3_sleep() as it is simple, yet a powerful addition to the testing. What this means is, as the state machine is going through all the possible combinations, one option is for it to skip ahead in time, eventually ending the crowdsale. If there is some combination of buying, skipping through time or trying to finalize early that causes an error, it will show the exact sequence of steps taken and accounts used to recreate the error.
Then next step would be to wrap the state_machine in a function, just as you would in a regular test, and let hypothesis do its magic!
#/tests/test_crowdsale.pydef test_state(crowd_sale, accounts, state_machine): # This is how many steps and examples you want to try
settings = {"stateful_step_count": 20
, "max_examples": 50} state_machine( StateMachine
, accounts[0:3]
, crowd_sale
, settings=settings)
It will then start to process a bunch of permutations, mixing different rules and checking if it broke the invariant. The moment I first got it running it had a bug and showed a step by step, each transaction how it broke the invariant.
Another cool example of a state machine using hypothesis is this: Solving the 3l jug puzzle from Die Hard 3.
Combining both property based testing and state based testing is crucial for complex code that deals with state change over time. Now 30% better with Brownie, pytest, hypothesis and time travel!
State based testing is a powerful tool essential for developers to think about as part of their testing/CI, as well as for preparing for, and running security audits on smart contract code.
If you’re going to deploy on mainnet, please do some testing and share your test suite before locking up other peoples money. Users will both have increased confidence in your contracts, and best case, developers will highlight errors in your test suite.
Thanks for reading. Below is my half baked example of Brownie on a project I’m currently building. Tests are in the /tests directory. I would appreciate any feedback in the spirit of open source and collective collaboration.
Big thanks to the super human Ben Hauser and all the contributors of Brownie for making such a joy of a tool for testing. Keep up the great work!
