Interruption of Service — Incident Analysis
Summary of events until today
On September 16, 2020, we were informed by a white hat hacker and our auditors that they had identified a bug in the “BondMaker” program, which is an integral part of the Lien app.
In coordination with various parties, including the white hat hacker and our auditors, we retrieved the users’ funds from the program and moved them to safety.
Once we confirmed that the users’ funds were no longer at risk, we shutdown the frontend of the Lien app and notified the community of the incident.
We immediately began tabulating everyone’s balances and made all efforts to return their funds as soon as possible.
On September 19, 2020, at 21:45 UTC, we completed the refund payments to users.
No funds were lost throughout the process.
What happened before and during the interruption of service
Before
A white hat hacker and our code auditors notified us on September 16, 2020, around 6:00 UTC that they had identified a bug in one of our programs.
Through coordinated effort with various parties, including the white hat hacker and our auditors, we took measures that were necessary to prevent the bug from being exploited and putting the user’s funds at risk.
Details of the actions taken are explained in this article.
During
At 12:20 UTC, we announced to users of the Lien app that a bug was found in the program and that measures were being taken to protect users’ funds.
The important facts that were communicated in the announcement were as follows:
- The assets (ETH) that have been deposited into the protocol are safe.
- No funds have been compromised.
- Addresses that held iDOL, LBT, SBT, and LP shares on September 16, 2020, at 11:07:43 UTC will be getting back their funds in ETH.
- The amount of ETH they will be receiving will be based on the values of the iDOL, LBT, and SBT calculated by the protocol as of that cut off timing.
- The payments will be processed as soon as possible.
Considering the nature of the matter and the urgency involved, we could not make the announcement until we made sure the funds were safe.
We determined that the most important thing was to protect users’ funds and prevent the exploitation of the bug by bad actors.
On September 17, 2020, at 7:40 UTC, we clarified that refunds will be calculated using the following Chainlink oracle ETH-USD exchange rate:
Cutoff timing: September 16, 2020 11:07:43 AM UTC
ETH-USD: $361.41
Chain link ID: 4782
https://etherscan.io/address/0xF79D6aFBb6dA890132F9D7c355e3015f15F3406F#readContract
On September 19, 2020, at 15:56 UTC, we announced that we had started processing the refund payments to users.
At 18:45 UTC, we announced that we completed the refund payments to users.
All funds that users held at the time of service interruption have been returned.
No funds were lost.
Technical Details of the Bug
There is a smart contract “BondMaker” that issues bond tokens (e.g. SBT, LBT, etc.) from deposited Ether and returns 1 Ether upon maturity for each group of bond tokens worth 1 Ether (“BondGroup”).
The contract is designed to redistribute Ether it has received to the bond token holders upon maturity while ensuring sufficient backing for the tokens issued in the process.
Here, a vulnerability was found where one could create a BondGroup with no cost by providing an empty array when defining its data structure, which allowed for creating an “empty” BondGroup with no validation errors.
While no SBT/LBT can be minted out of this empty BondGroup directly with issueNewBonds function, one could actually mint another BondGroup that consists of non-empty bond tokens (i.e. SBT and LBT) or exchange to any amount of collateralized Ether with the other exchange functions.
This could be achieved as follows:
A hacker could set the maturity of the “empty” BondGroup to that of a valid BondGroup — — specifically BondGroup 10 (i.e. BondGroup with id of 10) worth 25,647 Ether. This, in turn, made it possible for the BondGroup in question to be considered valid, which basically meant that the corresponding bond tokens that served as collateral could be created out of thin air.
The hacker could then exchange the empty BondGroup with a non-empty, valid BondGroup through the exchangeEquivalentBonds function, which allows for exchanging a BondGroup with another BondGroup that has the same collateral value.
To prevent potential exploitations by malicious actors, a collection of SBT/LBT worth 30,000 Ether was created through the process described above. It was then sent to the reverseBondToETH function and converted back to Ether, part of which (25,647 Ether) was utilized to back up the value of BondGroup 10.
What went wrong
Security has been our top priority since day one.
Lien smart contracts were audited not by one firm, but by two firms that are well known and trusted in the industry.
We made sure that each time we made changes to the audited code, the changes were verified by the auditors before they were deployed.
The code that has been deployed to the Ethereum mainnet is the same code that has been audited and no changes or additions have been made since they have been audited.
Code audits are an important part of assessing the validity and security of the code.
However, audits do not eliminate risks completely, as this incident has reminded us.
We have not taken this lightly and will be remediating and improving our internal control procedures over code development.
What we will be doing to prevent this from happening again
Improving security is an iterative process and we will continue to improve where we can.
In addition to the procedures that we currently have in place, we will be adding further layers of checks to mitigate risks that a significant bug is introduced into the system without being caught.
Internal Controls
Perform an extensive “what could go wrong” analysis involving all team members and external security experts before creating new code and functions.
Internal team members that are not working on the smart contract code will review the smart contract code as a second line of defense, before the code is submitted to external auditors for review.
Third party security expert involvement
We will involve third party security experts throughout the development process to ensure that attack surfaces are minimized and that sufficient consideration has been made from multiple angles.
Audits
As we have done since day one, all code that we will be deploying to the Ethereum mainnet will go through a full audit by independent auditors. We will also have auditor involvement during the testing phase which will improve the effectiveness and efficiency of the audit.
Monitoring Controls
Set up 24/7 monitoring controls to detect suspicious activity and implement reaction procedures in case of actual threats.
Next Steps
LIEN airdrop to users from team funds
To compensate users for the inconvenience caused and opportunity cost during the three days to which they lost access to their funds, we will be airdropping 3 LIEN to each account that held iDOL, LBT, SBT or LP shares in one of the FairSwap pools when the service interruption began (September 16, 2020 11:07:43 AM UTC).
The funds for this airdrop will come out of team funds (tokens allocated to Devs) and not from the community funds.
We plan to complete the airdrop during this month (i.e. September).
What will happen to the LP bonus boost payout?
The payout of the LP bonus boost will be calculated based on the liquidity provided until the service interruption. The full bonus pool will be paid out (i.e. 1,000 LIEN).
The payout is scheduled to occur during this month.
We will be announcing the details once they are determined so stay tuned.
When will the Lien app come back on line?
We will be rebuilding the Lien smart contracts from the bottom up to make sure that there are no other security vulnerabilities.
We will also be taking this opportunity to make improvements to some aspects of the protocol and UX/UI of the app, based on feedback that we have received from the community since the launch of the Lien app.
Some improvements that users can look forward to include:
- better gas costs
- improvements in overall UX/UI
- easier on-ramps of different tokens
As noted above, we will be involving third party security experts from the start and will be engaging an audit firm to perform a full audit of the new code.
This will be a major upgrade and it will take some time.
Different functions will be gradually released over the course of a 2–6 month timeframe.
This may seem like a long time, but when the entire Lien App V2 is released, we know it will be worth the wait.
During that time, we will continue to engage with the community so please send us ideas on what you want to see in V2.
We will continue to put out articles on our blog, going deep into functions that we haven’t had a chance to cover yet in detail (like auctions and the mechanisms behind that tranching of ETH).
We will be working hard the next few months to bring you Lien V2 which will be stronger and better than ever.
We thank everyone in the Lien and DeFi community for being supportive during this incident.
