Kintsugi Released Urgent Security Patches
Because of Kintsugi, our mission to bring Bitcoin to DeFi is stronger than ever
Postmortem Report on Kintsugi’s Critical Bug Fixes
Since the beginning, the purpose of Kusama and the canary networks in the Polkadot ecosystem was to break things. We encourage our ecosystem to break as many things as possible on Kintsugi, so when Interlay goes live, it’s bulletproof.
This past week, Kintsugi proved it is the valuable tool it was built to be. Critical bugs are never good news, but if the past few months have taught us anything, it’s that we have canary networks to avoid critical security issues in the future when it matters most.
Kintsugi is a ceramic art form that takes broken pieces and patches the weak points with gold, making something totally new and stronger. Through the chaos, there is order. By patching these bugs, we are more confident than ever in what we’re building. We will carry over what we’ve learned here into Interlay, improving our protocol. Because of Kintsugi, our mission to bring Bitcoin to DeFi is strengthened by the essence of the original blockchain itself: gold.
Here is the detailed postmortem of the critical issues that were found along with their fixes.
- Two critical bugs were reported via the Immunefi bug bounty program
- Within 24 hours, the bug reports were confirmed, fixed, and deployed via fast-tracked referenda
- Kintsugi was affected by the vulnerabilities. No funds were lost during the vulnerable periods.
- The Interlay network was not affected as the impacted code is not yet live.
- Both critical vulnerabilities affected the Vault theft reporting mechanism, which was deactivated as a resolution.
- A proposal will follow to upgrade the runtime to the next release and let the community vote if the removal of theft reporting should be made permanent.
Both bug reports were submitted by PwningEth.
Vault Theft Reporting
Vaults have custody over BTC and are not allowed to move the funds outside of predefined activities like processing of redeem, refund, or replace requests. The Vault client acts as an off-chain relay and scans other Vaults for potential misbehavior.
Report 1: Double Payment Forced Liquidation
The Vault client watches if other Vaults execute a particular request more than once. A Vault might move BTC by fulfilling a redeem request more than once. This particular behavior is reported in a function called `report_vault_double_payment`.
This function takes two BTC transactions as arguments to check if both transactions are unique. However, the implemented check only compared the two raw transactions, i.e., a hex-encoded BTC transaction.
This check was insufficient as an attacker was able to append random bytes to one of the two raw transactions to circumvent the uniqueness check.
Any Vault in the Kintsugi network that had at least one refund, replace, or redeem request processed could have been flagged for theft by anyone executing the attack. The attacker would be able to steal 5% of the Vault’s collateral. 150% of the liquidated BTC value would have been slashed from the attacked Vaults.
The problem was fixed by checking for unique transaction IDs instead of comparing raw transactions.
- Commit: https://github.com/interlay/interbtc/commit/3ef1476649697d6f7bbf785e39882ad965bd2ba8
- WASM Upgrade: https://kintsugi.subsquare.io/democracy/referendum/45
Report 2: P2SH Address Forgery
The Vault client watches if another Vault is moving funds outside of refund, redeem, and replace requests as well as if a Vault is paying out refund, redeem, and replace requests more than once. The reporting of the two activities. I.e., flagging Vaults for theft is done via the `report_vault_theft` and `report_vault_double_payment` functions.
When accusing Vaults of theft, both functions ensure that the BTC transaction with the alleged theft is confirmed on the Bitcoin blockchain. Once this is confirmed, a parser function extracts the BTC address of the Vault and checks with all BTC addresses stored on-chain for this Vault client if the address the BTC was spent from belongs to the Vault.
The parser is incompletely implemented and only accepts a subset of valid addresses. An attacker is able to utilize the incompleteness of the parser to correctly parse P2SH and P2PKH addresses. There are two attacks that can be executed:
- Avoid being flagged for theft by preventing extraction of the BTC address: A Vault can craft a BTC transaction that hides its address from the on-chain parser. That way, a Vault could commit theft, i.e., moving BTC, without other Vaults being able to successfully report this behavior.
- Force other Vaults to be liquidated: Similar to the first attack, a Vault can craft a BTC transaction that pretends to come from another BTC address. This attack works by crafting a P2SH transaction that looks to the parser like a P2PKH transaction. The Vault would insert a victim signature and victim public key before its actual signature to be able to report the victim Vault for theft. An example for such a transaction is provided here: https://blockstream.info/testnet/tx/e0d2457036493db0787636a926b543f96dbfca157a6ca81d5fb97f2cc70e1e5b?expand
Any Vault in the Kintsugi network that had at least one refund, replace, or redeem request with a P2PKH or P2SH transaction associated could have been flagged for theft by anyone executing the attack. The attacker would be able to steal 5% of the Vault’s collateral. 150% of the liquidated BTC value would have been slashed from the attacked Vaults.
The vulnerability was fixed by deactivating theft reporting (functions `report_vault_theft` and `report_vault_double_payment`). The deactivation of the functions was done by overwriting them to not change storage but leaving the functions in to not require Vaults to update their clients on short notice.
- Commit: https://github.com/interlay/interbtc/pull/667/commits/744ba218a019e7108cf29555ad81b05977fe8399
- WASM Upgrade: https://kintsugi.subsquare.io/democracy/referendum/47
Removing Theft Reporting
Due to the large attack surface of P2SH parsing, we made an emergency fix to deactivate Vault theft reporting. This is a change to the core protocol with the following implications:
- Economic Security Unaffected: Removing Vault theft reporting has not impact on the economic security of Vaults or users. When users request to redeem kBTC/iBTC for BTC, the Vault is required to have the necessary BTC available as it stands to lose its collateral should the request not be fulfilled in time.
- Simplification of Vault Operation: We had incidents on Kintsugi where Vault operators were flagged for theft without malicious intent. Rather, due to Bitcoin congestion or operational issues, Vault operators got liquidated. Removing theft reporting will prevent such cases from happening in the future.
- Amount of BTC Locked may not be the same as total issuance of kBTC and iBTC: The economic security of the Kintsugi and Interlay networks remains unchanged as Vaults still need to provide the same amount of collateral to secure kBTC and iBTC. However, Vaults are now able to move BTC at any time. Thus, the amount of BTC locked by Vaults currently might not be the same amount of kBTC or iBTC issued at all times. Rather, Vaults now need to ensure that BTC is available when users want to redeem, but they can move BTC elsewhere if they wish so.
- We will open a governance proposal to discuss the implications of this change in more detail.
Security and Audits
The affected code has been audited before and has been used in production since the launch of kBTC. While we are taking utmost consideration towards the security of our code, these two vulnerabilities were left undetected prior to their reporting via the bug bounty program. We are taking the following learnings and next steps:
- Moving parsing code off-chain: We have re-implemented large parts of the Bitcoin core code-base in our on-chain relay. While we had a discussion for a long time to move parsing off-chain, we are now convinced that this should happen soon. By ensuring we only pass already parsed types to the on-chain logic, we are able to reduce the attack surface.
- Further audits: In addition to the existing four audits (three published here https://github.com/interlay/interbtc/tree/master/docs/audits), we have further audits scheduled with Quarkslab as well as SR Labs this year to ensure that our code including its latest changes is fully audited.
- Success of the bug bounty program: Collaborating with Immunefi on the bug bounty program has proven to be a big success. While it is never great to have critical vulnerabilities, we have found a good way for whitehat hacker to report them via the Immunfi program. We will continue to collaborate with whitehat hackers to confirm and resolve any reported vulnerabilities.
The Polkadot ecosystem created Kusama as a vessel for contained chaos and experimentation. Canary networks test the air for us before we delve into the coal mine, and unfortunately, sometimes they get hurt. However, mistakes are necessary for growth. Patching weak points on Kintsugi will only make us stronger on Interlay.
We’re grateful for the immediate success of the Immunefi Bug Bounty program and will continue to look towards the community of whitehat hackers to break things. We’ve learned a lot.
As always, we will be transparent with our community every step of the way, moving forward with decentralized, trustless governance.
We envision a future where blockchains seamlessly connect and interact: anyone can use any digital asset on any blockchain, trustless and without limitations. Interlay works with Bitcoin, Ethereum, Polkadot, Cosmos, and others to expand interoperability, capital efficiency, and openness.
Our flagship product is interBTC — Bitcoin on any blockchain. A 1:1 Bitcoin-backed asset, fully collateralized, interoperable, and censorship-resistant, realizing the true free nature of BTC and decentralized finance.