The Security Series: Have Smart Contracts Become More Secure Over Time?
A first look into the strength of the smart contract layer of Ethereum measured by SWC.
In this article, we describe how we have run all distinct Ethereum bytecodes through the contract security analysis framework MythX. From this data collection, we recorded the detected weaknesses, classified them, and analysed the results. We therefore plot the safety trend over time on Ethereum, measured by daily average number of detected vulnerabilities, across deployed contract bytecodes.
Smart Contract Weakness Classification (SWC) Registry
The Smart Contract Weakness Classification Registry (SWC Registry) is an implementation of the weakness classification scheme proposed in EIP-1470. The goal is to provide a straightforward way to classify security issues in smart contract systems.
Currently, there are 33 weaknesses listed in the registry. Details can be found at:
- SWC Registry: Smart Contract Weakness Classification and Test Cases
- Smart Contract Weakness Definition
- All smart contract security issues in one place: An introduction to the SWC Registry
Dataset
The Ethereum bytecode is an assembly language made up of multiple opcodes. Each opcode performs a certain action on the Ethereum blockchain. While developers build smart contract functionalities using high level language like Solidity, the Ethereum Virtual Machines compile them into low level language as bytecodes to execute actions.
Out of 33 defined patterns of smart contract weaknesses in the registry, some are related to Solidity syntax. Thus, in the following experiment, we only scanned for 15 SWCs that apply specifically to bytecodes and, therefore, to the smart contracts that employ those bytecodes. Note that MythX is backed by assumptive algorithms drawn from potential attacks, and run by machine without manual verification. Thus, there is a chance of false positives being captured in the results.
Each bytecode can be executed in multiple smart contracts on Ethereum, meaning many contracts are similar — including with respect to their SWCs. After scanning every smart contract deployed on mainnet and eliminating duplicate bytecodes, we are left with 197,355 unique bytecodes. The most commonly-deployed bytecode can be found in ~1.5million contract addresses on Ethereum.
The analysis in this article centers on those ~200k unique bytecodes that each (can) include multiple SWCs, and which have been deployed to multiple smart contracts across the Ethereum network.
Safety Analysis Over Time
Our analysis looks at all the bytecodes ever created over Ethereum’s history. We wanted to answer the question: have smart contracts become more secure over time?
Averaged over all bytecodes created on the same date, we see the distribution of the average total number of SWCs detected from bytecodes deployed over time. Note that extreme outliers are excluded to avoid influence from false positives.
It shows three stages of smart contract security with respect to mean and variance.
- “The Wild West”, June 2015 — April 2017 (dark gray): It’s the early days of Ethereum and for solidity developers. With the launch and initial growth in the developer community, the quality of contract codes varies largely, without much experience and guidance yet in security. Thus we see the sample fluctuates dramatically, with the initial batch of bytecodes deployed with a high variance (vertical distribution), after which a converging trend emerges, coinciding with the maturing of Ethereum.
- “The First Hype”, April — September 2017 (light pink): In the middle of 2017, we see the daily average count of weaknesses from newly created contracts spike, with a small variance. Smaller variance means data points are closer to each other, indicating that the increased number of weaknesses was the major trend during that period. After manually validating the purpose of the contracts created during this time, we believe most of them are for investments in crowdsales. After all, 2017 was the year of ICOs.
- “The Long Road to Excellence”, September 2017 — now (white): The red trend line is fitted with linear regression onto the sample. As time goes by, the total number of newly detected SWCs decreases monotonically. We believe this indicates that, with rising awareness and efforts for contract auditing & security, the quality of deployed code is steadily increasing over time and the ecosystem matures.
What’s More?
We’re not done with Ethereum security case studies. Besides SWCs, there are other aspects of security to be investigated. In the security series, we plan to dive into a variety of metrics to build a holistic view of “risk”, including:
- Contracts balance holdings of coins and tokens,
- Invalid messages and failed transactions,
- Multiple times of creation and destruction.
Let us know any comment and interesting path in your mind!
About MythX
Built by a team of security experts, MythX is the premier security analysis service that automatically scans for security vulnerabilities in Ethereum and other EVM-based blockchain smart contracts. MythX’s comprehensive range of analysis techniques including static analysis, dynamic analysis, and symbolic execution, can accurately detect security vulnerabilities to provide an in-depth analysis report. With a vibrant ecosystem of world-class integration that amplify developer productivity, MythX can be utilized in all phases of smart contract development lifecycle. Visit MythX.
