Post-Mortem & Remediation Plan
Incident Summary
On Jan-04–2024 at 3:42 AM +UTC, members of Gamma noticed a tweet indicating an exploit implicating Algebra Finance. Being that Gamma services many exchanges built on Algebra’s infrastructure, we immediately took action to investigate the issue.
After a quick investigation and hearing reports from some users regarding their funds, we realized that Algebra had been erroneously implicated in the exploit, and that select Gamma vaults were being exploited.
The attack had a recognizable pattern of attacking our stablecoin and pegged asset (LST) vaults on Arbitrum across Uniswap, Camelot, and Ramses. At that point, we made the decision to shut down all deposits to our vaults.
Response
The first response we took was to immediately restrict vault deposits to mitigate the attack vector. Because the attack vector relied on the ability to make deposits into the pool, restricting vault deposits on every public-facing vault was a broad but effective means of nullifying the attack any further. Going network-by-network, we restricted all deposits at the smart contract level and notified our partner AMMs as soon as such actions were taken.
Gamma also joined a “war room” chat, where we could communicate and seek help from various partners, security professionals, and stakeholders. This group was invaluable in keeping a clear line of communications with our community.
Gamma then posted a series of tweets updating the community about the exploit.
Exploit Costs
The compromised vaults are indicated below.
gDAI-DAI 0.01% (Uniswap — Arbitrum)
Vault Address: 0x33985Ca762541e2412F454c6F2e7EC677645D1dF
Pool Address: 0x9F934D552476c992De3751A4873f2C1fBEB032B2
Losses: ~ $2.74M
wstETH-WETH (Camelot — Arbitrum)
Vault Address: 0x3D53aC3Abec01827cAaE5Bc934d46b171cEa2206
Pool Address: 0xdEb89DE4bb6ecf5BFeD581EB049308b52d9b2Da7
Losses: ~ $771K
USDT-USDC.e (Camelot — Arbitrum)
Vault Address: 0x61A7b3dae70D943C6f2eA9ba4FfD2fEcc6AF15E4
Pool Address: 0x3AB5DD69950a948c55D1FBFb7500BF92B4Bd4C48
Losses: ~ $1.357M
USDC-USDC.e (Ramses — Arbitrum)
Vault Address: 0x80709a760Ff54112bD3e0CE31C104d912bA51774
Pool Address: 0x562d29b54d2c57F8620C920415C4dCEAdD6dE2d2
Losses: ~ $1.313M
Total Losses: ~$6.18M
Exploit Methodology
The exploiter exposed a defect in our deposit proxy configurations to exploit the aforementioned stable and LST vaults.
Our vaults typically have four main sources of deposit protection against flash loans:
- Mandating a ratio of token0 and token1 in accordance with the ratio in the pool
- Setting a price change threshold, such that deposits will be disallowed when price change exceeds a certain amount from the TWAP oracle price
- Setting deposit caps per deposit
- Disallowing single-sided deposits
The main issue has to do with the deposit proxy settings we placed on (2) the price change threshold. It was placed too high, allowing for a -50% / +100% price change on certain LST and stablecoin vaults. This allowed the attacker to manipulate the price up to the price change threshold and mint a disproportionately high number of LP tokens.
The mechanism behind the attack, which all involved the same pattern was as follows, using the gDAI-DAI vault as an example:
- At block 166874977, the exploiter began a series of attack transactions utilizing flashloans from Uniswap and Balancer to borrow funds to manipulate the pool price of gDAI and exploit the faulty deposit configurations by looping the deposit and withdraw functions of the gDAI-DAI vault continuously
- The attacker would inflate the pool price of gDAI, and deposit gDAI at the inflated price and receive a disproportionate amount of LP tokens
- The attacker would loop this process continuously across 5 different transactions using this address here: https://arbiscan.io/address/0x5351536145610aa448a8bf85ba97c71caf31909c
Why Did This Happen?
The main source of the error was due to an arithmetic error in our automation scripts when configuring deposit safety settings. The vault smart contract would not have been susceptible to this attack had the deposit configurations been set correctly at the intended 2% price threshold.
On September 14th, we implemented new safety standards on all stable and LST vault pairs that entailed longer TWAP intervals and slightly higher price change thresholds. Prior to this implementation, all pairs, including volatile, stable, and LST pairs, had the same deposit configurations, and we felt we needed more conservative deposit proxy configurations given the tight ranges that were being set on these pairs. In our simulations, we deduced that a 2% price change threshold was reasonable to use for the longer TWAP intervals.
While it was our intention to provide additional security, the error in our automation scripts erroneously applied price change configurations that allowed for an unintended -50% / +100% price change versus the intended 2% change. It was precisely this error that allowed the attacker to manipulate the price higher than intended which led to the loss of funds.
Mitigation
We are engaging OpenZeppelin for an audit of our deposit proxy configurations. OpenZeppelin has a leading smart contract security audit team, and we owe it to our partners and users to go the extra mile in ensuring the safety of our contracts.
The engagement is expected to start early next week, after which we will implement the changes necessary to ensure the safety of our deposit proxy configurations for all our vaults. Once the changes have been implemented, we will reopen deposits. The estimated time of re-opening deposits could be anywhere from 1–3 weeks.
Fund Recovery Attempts
On Jan-04–2024 at 10:54:47 AM +UTC, Gamma sent an online message via Etherscan from our Gammastrategies.eth wallet to the Gamma Exploiter. A similar message was also sent via Arbiscan. The message read:
“Hello. We are reaching out to you from Gamma. We noticed that you found a bug, and we want to thank you for discovering it. We hope you did so as a whitehat or grayhat. We want to open up communications regarding bug bounty. Please email us at: gammastrategies2@protonmail.com”
https://etherscan.io/tx/0x293698c1ab8b7c411d17aff9176c60ebafbeddefe557ff80f8dddd50c77e2cc2
Since publishing this report, Gamma is reaching out to security professionals to see how it can maximize the potential for fund recovery.
Remediation Plan
We are absolutely heartbroken that users lost funds due to this miscalculation, and we are determined to do right by our users by compensating them in full for their losses.
Our plan for compensation will be the following:
- All affected users and their values lost will be posted here: https://docs.google.com/spreadsheets/d/1rrGlphQutOCFA4M4JVmcGBIBiclPYLx7LEES5ENrM6I/edit#gid=0
- We will allow for a one week dispute period for anyone to dispute the amounts lost or the inclusion/exclusion of any user address. Please contact us in our Discord if you see any issues
- We will then fund a Recovery Pool and all affected users will receive a proportionate share of their recovery in USDC
The amounts sent to the Recovery Pool will be based on our current annualized revenues, which annualize the past 30 days. We are currently operating at annualized revenues of $8.97M which is the category indicated in the red box below. See the following chart for the revenue split mechanism:
To help the affected users monitor a potential recovery, we have posted our annualized revenue stats on our frontend here. The revenue splits will be conducted on a biweekly basis and reassessed every two weeks.
Moving Forward
As a team, we will be placing a much greater emphasis on safety, in all areas of the code, including both smart contract and offchain scripts. Going forward, we will have much more thorough checks and review procedures to ensure this mistake is not repeated.
We understand that anything short of immediate full recovery may be a disappointment to those affected; however, we are absolutely determined to do right by our users and partners in providing a remedial solution that will provide a full recovery and ensuring public confidence in the Gamma product through a full OpenZeppelin audit. We had astronomical growth in 2023, and Gamma is determined to become even stronger, smarter, and safer than before to ensure the quickest recovery possible.
As a final note, we want to thank all our users, partners, and stakeholders for sticking with us through the most difficult of times. Words cannot express how grateful we are for the outpouring of support we received. The team is more motivated than ever to continue the success of Gamma.
Other FAQs
Is it safe to keep funds in the vaults? Are they safe from this attack vector?
All funds in the vaults currently are immune from the attack vector because all deposits have been shut down, and the attack vector depends on making a deposit into the vault. Therefore, all funds in vaults now are safe.
Are withdrawals from the vaults possible?
Yes, only the deposits were halted. Any current LP may permissionlessly withdraw their position, but they will not be able to re-deposit until the deposits are opened.
When can we expect deposits to reopen?
We expect optimistically be open in 1 week and conservatively to be open in 3 weeks. We will update the community on the status of our audit engagement with OpenZeppelin which is expected to start early next week.
Has anything happened to the GAMMA token or xGAMMA staking?
No, this incident was solely related to the aforementioned vaults.
Is Gamma pausing the integration of new AMMs or chains?
As soon as deposits are re-opened, we will continue the integration of new AMMs and chains. The deposits will be re-opened after our OpenZeppelin engagement which will begin early next week.
Are Gamma’s hypervisors/vaults going to need replacement? Will new vaults have to be launched?
Because this exploit was related to the deposit safety configurations, the vaults will not be replaced, but the configurations will surely change.
Why not compensate all the victims immediately?
The total loss was $6.18M, and we simply do not have the funds to cover all of that right now. However, we believe our current compensation plan will allow us to sustainably grow the protocol while providing a timely recovery.
How much will initially be going to the Recovery Pool?
After the one-week dispute period of our list of affected users here, we will fund the Recovery Pool with the aforementioned revenue distribution plan as well as the proceeds from 120k USDC of our own funds.
When can affected users expect to be fully reimbursed?
Assuming no growth or decline in our current revenue numbers of $8.97 million, it would take us approximately 1.73 years to make all users whole.
We are confident that once deposits are reopened, we will be a much safer and stronger protocol, and we fully expect to return to growth this year.
