Building a Lottery, Part 2

Otherwise known as, “How to unit test?”

When I approach building something like a contract on Ethereum, there are two key elements that I keep in mind:

  1. I can’t edit the code after it’s deployed
  2. I need to be able to deploy upgrades in the future

The latter is more of a design concern, covered in the first part of this series, “Building a Lottery”. The former, however, encourages me to lean heavily on the ability to unit test various pieces of code and how they fit together.

Building a smart contract — especially a set of them — is a lot like putting together one of those mechanical puzzles. There are a lot of strange-shaped pieces, but once they start falling into place, they turn into something that looks and feels solid.

In order to build a mechanical puzzle like this, each piece must be a unique shape, must be carefully machined, and must be planned in advance to move smoothly together so as to get everything into place in the right order.

In the smart contract world, ensuring that each piece of your puzzle is made to the exact specifications is what tests are good at.

Unit testings roadblocks

For this particular contract, I used embark, which let me get set up pretty quickly on writing unit tests. I started with the boilerplate that comes pre-generated with embark new, and worked my way up from there. I started with the LotteryRound contract (as it was when I began writing tests), intending to work “outward” from there.

Roadblock 1: inconsistent hashes

Unfortunately, it didn’t take long until I ran into my first snag. See, it turns out that web3 and Solidity handle inputs to sha3 entirely differently. For the lottery, this turns out to be a pretty big problem. If you can’t generate the same hashes on both “sides” of a unit test (e.g. in javascript land and in the EVM), it becomes rather difficult to be sure your contract is doing what you want it to do — or that you’ll be able to prove you generated some hash outside of the contract later on.

The linked github issue provides a possible solution, but it, too, turns out to be flawed. First, it requires transpilation from ES6, something I wasn’t able to divine how to do in the embark environment. Second, it leaves the leading 0x on every piece of information being encoded.

So, as is my style, I rolled my own solution.

Roadblock 2: the rules of the game

This second roadblock was a little bit of my own doing. To reduce the scope of features I needed to implement, I decided to hardcode all of the initial values for the LotteryRound contract: round length, ticket price, etc. However, this removes the flexibility to create short-lived rounds for testing purposes.

So, I needed a way to get around the rules in the testing world, but in such a way as to prevent myself from leaving “cheat codes” in the final product.

Enter DebugLotteryRound.

Roadblock 3: Endless boilerplate

This one is more of a developer quality-of-life complaint more than anything else. With web3, it’s easy to get into callback hell, so my natural response is to use Promises. However, I also need to ensure that I’m writing tests that can be run by others, to independently verify that they’re indeed testing what they say they’re testing.

This leads to the interesting compromise that is Promisify, declared near the top of every test file. This little shim lets me pass in a bound (e.g. curried) web3 or contract invocation and get back a promise that fulfills when the request is complete, or rejects on an error.

The second bit of endless boilerplate was the constant contract.method … web3.eth.getReceipt pairing needed to ensure that transactions had successfully been mined. This resulted in the creation of a myraid of small helper functions to invoke the various contract methods I was testing

You can see how these all fit together in the massive test that is my integration test.

Roadblock 4: framework limitations

Some of the things I needed to do with the contracts to test them simply weren’t supported, or had bugs. For example, if I attempted to deploy both a LotteryRoundFactory and a DebugLotteryRoundFactory, embark would somehow silently fail to deploy either one.

This led to a few contributions back to embark, in the form of better testing options and the ability to use custom configurations — as well as some changes that I was unable to incorporate back into the mainline due to it also undergoing a large refactor.

Unit testing methodology

Having worked through the above blocks and more, I realized I had coded myself into a bit of a corner in a few places, as I initially started off testing behaviors, rather than units of code. This led to tests that were disorganized, fragmented, and likely full of gaps in coverage.

So I undertook a rather large refactoring of tests, to switch to a purely unit-driven approach: each method would get its own set of tests, relying on behaviors as little as possible. This let me set better expectations around what each contract method could and could not do, and should and should not do. The refactor to a unit-first approach also revealed a plethora of small bugs in the contracts, such as backwards logic in the paidOut function, or a != instead of an == in the isUpgradeAllowed function.

Finally, by leveraging the Debug... contracts in the unit tests to test otherwise long-running behaviors, I was able to reach full coverage of the behaviors I was writing in Solidity. This further enabled me to strip down and slightly reorganize the contracts themselves, so that they fit together more smoothly and with fewer rough edges.

End result

The results I’m actually quite happy with. I have absolute confidence that the contract behaves as I want it to. I’m not confident I designed the behaviors correctly, per se, but I’m confident that the behaviors I wrote are the ones that the contract enforces.

Testing only goes so far, however. There have been a couple lively discussions around the nature of the PRNG and various means of attacking it — things, it should be noted, that fall outside the scope of unit tests. It’s important to remember that tests don’t provide any guarantees that your solution is the best solution — only that your chosen solution is indeed implemented correctly.

Next up: deployment, and the mixups that arise when you don’t have or use an integrated deployment mechanism.