Acala Incident Report — 14/08/2022

Published in

Acala

4 min readSep 20, 2022

A misconfiguration of the iBTC/aUSD liquidity pool (“iBTC/aUSD LP”) (which went live earlier the day of August 14, 2022) resulted in error mints of a significant amount of aUSD. The reward for staking of iBTC/aUSD LP tokens accorded to contributors to iBTC/aUSD LP was intended to be INTR and ACA as outlined in the community post here announcing the launch of the iBTC/aUSD LP.

16 iBTC/aUSD LP contributors, when claiming their iBTC/aUSD LP rewards, received these error mints. Some of these LP contributors repeatedly added more liquidity to the pool and claimed more aUSD error mints, resulting in more aUSD being erroneously minted. Some of these identified LP contributors exchanged error mints they received for other tokens including DOT, IBTC, INTR, etc, and they transferred aUSD error mints and error mint-swapped tokens to other XCM-connected chains such as Polkadot, Moonbeam, Interlay, Astar, as well as CEXs.

Swapping error minted aUSD with other tokens led to a disruption in swap ratios of aUSD with other paired tokens in the pools on Acala Swap. A number of other users acquired aUSD error mints at a significantly disrupted exchange ratio. Some of these users repeatedly swapped more aUSD error mints as the imbalance of pools grew. They then transferred a significant amount of aUSD error mints to other XCM-connected chains and CEXs.

Response

To prevent further error mints, swaps, and outflow of error mints, a number of urgent governance votes were passed to reset the misconfiguration and pause the incentives pallet, DEX, honzon protocol, oracle pallet, LDOT instant redeem, non-ACA token transfers, xcm transfer out, EVM+, etc.

A number of trace reports were published in the following days and weeks to reveal the full extent of the incident:

There were total 3.022B aUSD error mints
2.96B were found in the addresses of the 16 identified LP contributors (trace here and here)
12.38M aUSD error mints were found on the top 35 accounts that acquired a significant amount of aUSD error mints or were linked to the accounts that acquired a significant amount of aUSD error mints (trace here)
The remaining 52.068M aUSD error mints, error mint-swapped tokens and every address involved in the incident were identified in the full trace report here.

The community could then verify the information using on-chain data and formulate proposals to resolve the issues, re-collateralize aUSD, and formulate plans to resume services on the Acala network.

A bounty was offered (and still remains open) to those who returned aUSD error mints and error mint-swapped tokens that were transferred out to other parachains. See more details here.

The root cause was identified and verified by an internal peer review and an external security auditor. The fix has been audited (here) and is ready to be applied in the next runtime upgrade via a governance vote.

Vulnerability

The root cause of the incident was a vulnerability in the DEX saving code that is part of the incentives pallet. The incentives pallet is consist of two functions to accumulate incentives periodically on block initialization:

accumulate_incentives: accumulate incentive rewards (in various tokens) from the rewards vault where reward tokens are deposited
accumulate_dex_saving: accumulate aUSD rewards from honzon stability fees (which are then shared with aUSD-pool LP contributors)

The DexSavingRewardRates specifies the rate/amount of aUSD rewards. The erroneous minting of aUSD on Acala was caused by the misconfiguration of the DexSavingRewardRates parameter of the accumulate_dex_saving function, which should have prevented the misconfiguration from taking effect. The accumulate_dex_saving function is a deprecated feature that was only used when Acala’s canary network on Kusama, Karura, was first launched.

Soon after the incident, to prevent further error mints, the DexSavingRewardRates parameter was reset to 0 (as originally intended) and the incentives pallet was paused via urgent governance votes. Further investigation has revealed the above code vulnerability, which could be fixed by removing dex_saving related functions (accumulate_dex_saving and update_dex_saving_rewards) and configuration (DexSavingRewardRates). A runtime upgrade will be required to apply the fix.

Audits

Upon launch, Acala’s codebase has been audited by Trail of Bits, SlowMist and SRLabs. SRLabs also provides continuous security assurance for Acala by auditing ongoing changes (in the form of Pull-Requests).

Security Roadmap

There are a number of initiatives on the roadmap to further strengthen the security of the Acala network:

Parameter safeguard mechanism to continuously check and prevent misconfigurations
Pallet safeguard mechanism to isolate risks in individual pallets using proofs
Ongoing monitoring system for proactive detection of incidents

A report on the current state of the Acala network and recommendations for resuming operations will be published soon.

About Acala

Acala is a decentralized finance network powering the aUSD ecosystem. The core product, Acala USD, is a decentralized, multi-collateral, crypto-backed stablecoin serving as the native stablecoin of the Polkadot ecosystem. Acala’s Ethereum-compatible blockchain has built-in DeFi protocols for application developers to leverage, including a trustless staking derivative (liquid DOT — LDOT), a decentralized exchange, and the EVM+, a hybrid EVM offering fully Ethereum-compatible development environment plus full compatibility with Substrate. Karura is Acala’s sister parachain to serve the Kusama ecosystem and shares the same codebase as Acala.