The Security Series: Have Smart Contracts Become More Secure Over Time?

A first look into the strength of the smart contract layer of Ethereum measured by SWC.

alethio
alethio
Jan 3, 2020 · 4 min read

In this article, we describe how we have run all distinct Ethereum bytecodes through the contract security analysis framework MythX. From this data collection, we recorded the detected weaknesses, classified them, and analysed the results. We therefore plot the safety trend over time on Ethereum, measured by daily average number of detected vulnerabilities, across deployed contract bytecodes.

Smart Contract Weakness Classification (SWC) Registry

Currently, there are 33 weaknesses listed in the registry. Details can be found at:

Dataset

Out of 33 defined patterns of smart contract weaknesses in the registry, some are related to Solidity syntax. Thus, in the following experiment, we only scanned for 15 SWCs that apply specifically to bytecodes and, therefore, to the smart contracts that employ those bytecodes. Note that MythX is backed by assumptive algorithms drawn from potential attacks, and run by machine without manual verification. Thus, there is a chance of false positives being captured in the results.

Each bytecode can be executed in multiple smart contracts on Ethereum, meaning many contracts are similar — including with respect to their SWCs. After scanning every smart contract deployed on mainnet and eliminating duplicate bytecodes, we are left with 197,355 unique bytecodes. The most commonly-deployed bytecode can be found in ~1.5million contract addresses on Ethereum.

The analysis in this article centers on those ~200k unique bytecodes that each (can) include multiple SWCs, and which have been deployed to multiple smart contracts across the Ethereum network.

Safety Analysis Over Time

Fig. 1: Scatter Plot of Bytecodes’ Initial Deployed Time vs. Avg SWC Occurrences

Averaged over all bytecodes created on the same date, we see the distribution of the average total number of SWCs detected from bytecodes deployed over time. Note that extreme outliers are excluded to avoid influence from false positives.

It shows three stages of smart contract security with respect to mean and variance.

  • “The Wild West”, June 2015 — April 2017 (dark gray): It’s the early days of Ethereum and for solidity developers. With the launch and initial growth in the developer community, the quality of contract codes varies largely, without much experience and guidance yet in security. Thus we see the sample fluctuates dramatically, with the initial batch of bytecodes deployed with a high variance (vertical distribution), after which a converging trend emerges, coinciding with the maturing of Ethereum.
  • “The First Hype”, April — September 2017 (light pink): In the middle of 2017, we see the daily average count of weaknesses from newly created contracts spike, with a small variance. Smaller variance means data points are closer to each other, indicating that the increased number of weaknesses was the major trend during that period. After manually validating the purpose of the contracts created during this time, we believe most of them are for investments in crowdsales. After all, 2017 was the year of ICOs.
  • “The Long Road to Excellence”, September 2017 — now (white): The red trend line is fitted with linear regression onto the sample. As time goes by, the total number of newly detected SWCs decreases monotonically. We believe this indicates that, with rising awareness and efforts for contract auditing & security, the quality of deployed code is steadily increasing over time and the ecosystem matures.

What’s More?

  • Contracts balance holdings of coins and tokens,
  • Invalid messages and failed transactions,
  • Multiple times of creation and destruction.

Let us know any comment and interesting path in your mind!

About MythX

Alethio & MythX partnered to bring you this Security Report

Alethio

The blockchain is our collective memory.