If Code Should Be Law We Need Better Development Standards

On Smart Contract Architecture And Testing

The recently discovered bug in the ether.camp token handling smart contract (and any project that inherits this contract, basically all ether.camp hackathon projects) again highlighted the special needs for good architecture, testing and security in smart contract development.

The problem with this contract is pretty similar to what happened to the DAO — a smart contract with control of a value token has a bug (because of an oversight in its implementation) that allows an outsider to issue calls to it that have unintentional consequences.

Smart contracts are something new because for the first time we have computer code without governance that controls actual money, and in some cases serious amounts of money.

Why Smart Contract Development Is Special

Smart contract development is different from most other contemporary development as there are rarely any easy upgrade paths, especially in the critical parts (that control ether or other value tokens). Deploy it and once the contract is in use you are pretty much stuck with it.

If we learn from software architecture and quality assurance in other industries with very high security requirements or with difficult upgrade paths — aerospace, embedded systems or medtech (where I have my background) — I hope that we as a community can develop a clear minimum set of best practices in smart contract development that will help minimize the risks associated with contract development and deployment as much as possible.

Others that are more involved with ethereum development already have this started (e.g. see the the Zeppelin Smart Contract Framework). My goal with this post is to highlight this issue again and get an even higher level discussion going and hopefully be able to define a minimum set of architectural and testing requirements that all core smart contracts should be subject to.

Please give me feedback on Medium or Reddit and I’ll try to keep track of comments and write a Github wiki with the current list that can be shared and referenced to.

Architecture

For starters, I propose the following architectural best practice check list:

1. Use and reuse audited open source code as much as possible (such as ERC: 20)
2. Separate token issuance and token control from all other code into separate contracts
3. Implement a safety stop (kill switch) function at the top of the code in case something happens
4. Implement a contract transfer function whenever possible

Testing

There is an absolute need to follow current best practices in testing. This is an established field and I won’t go into the details, but basically there are two parts — verification and validation testing.

Verification is to verify that the code has been written according to the requirements and design specifications. At release (and preferably during development) this is tested in unit tests, functional test and integration tests. These tests can and should be automated and cover expected states and different fringe cases.

On top of the verification testing formal verification, code reviews and security audits should be done to further verify the code and find bugs, errors and weaknesses.

Validation, the other type of testing, has so far not been discussed to the same extent as verification in smart contract development. Validation ensures that the product meet the user needs, and that the requirements were correct in the first place. Usually this is done through validation with external users in alpha/beta-testing or in studies.

The DAO failure happened because of missing validation testing and the current ether.camp bug is the result of bad verification testing. Many times the flaws in complex code will only be visible when the code is actually used and there are real users and data involved, and with that real values at stake.

For example, validation testing in Ethereum can be done through early deployment on the testnet, and then later as proof-of-concept on the real network before being deployed and given it’s full rights on the chain. Validation testing can also be done in many other forms, e.g. using the proposed contract in a production setting but with limited risk compared to the future full deployment.

Bear in mind that validation is to ensure that the code works as intended by its creators. Code that does not behave as intended is simply buggy code that is not validated and should never have been released in the first place.

If code is to be law as some argue, then it must be properly verified and validated — and therefore work as intended before being released. As long as code is in validation I see absolutely no problem with using any tools available in order to update the code and fix the bugs.

Two great examples of validation testing that I would like to highlight is Ethereum and Rouleth.

Ethereum itself still undergoes validation testing. Some of it is intentional, releasing the production system early when ether had limited value, the increasing importance of a functioning backend/blockchain and size of the ecosystem, increasing number of transactions and users, slow introduction of new features and soft- and hard forking. Some is also unintentional, like The Hackers long time stress testing.

The creator of Rouleth (a gambling dapp) released the contract early (8 months ago) and has iterated the contract and slowly increased the number of bets and the stakes involved over time in order to validate the code. Releasing a proof-of-concept and over time test it with more and more users/assets/importance is a great way to validate as well.

My proposed minimum testing for smart contracts is:

1. Have a testing plan for both verification and validation testing
2. Do code reviews
3. Do verification testing with automated unit, functional and integration tests
4. Do external security audits
5. Do validation testing by extensively testing on the testnet and then
6. After testnet testing, do more testing with limited risk in proof-of-concept or alpha stage on the main net

Conclusion

This is by no means an exhaustive list of all quality assurance activities that should be done for smart contract development. Instead, it’s the start of a discussion. Please get back to me with ideas on how to improve this simple checklist for all smart contract developers out there to use.

But, if you are a smart contract developer or plan to use smart contracts in your project in any form and are unfamiliar with any of the concepts above — DO YOUR HOMEWORK FIRST. In most cases you will only have one chance to screw up.

Thanks to /u/frenchhoudini, /u/airbolab and Zack Coburn for highlighting these issues, and Manuel Aráoz at Zeppelin for his work on the Zeppelin framework.