Responsible Vulnerability Disclosure

Over the past week we have received two responsible vulnerability disclosures. None of them have been exploited and no mutual funds have been compromised. This article goes into the details of both disclosures, what we learned from them, and how we’re integrating these learnings into our practices.

Security is a critical aspect of what we are doing at Nexus Mutual. It’s at the core of our product and is absolutely essential if the mutual is to be successful now and into the future. We will always put the security of our protocol and the safety of our members funds above anything else. We encourage everyone to read this disclosure in full to understand what happened, what we have learned and hopefully as insight for other teams dealing with similar issues.

On Tuesday evening, February 18th, we received a report of a potential issue related to governance discovered by Mudit Gupta. After a careful analysis we confirmed the issue exists and impacts the Advisory Board and the Owner; unintentionally granting them additional privileges.

At Nexus Mutual, all governance actions are subject to Member vote, Advisory Board vote, or Owner vote. The proposals can be raised by either member type. However, member-raised proposals must be whitelisted by the Advisory Board. Each governance proposal has a category associated with it that defines the solution to execute when the vote is closed.

The underlying bug was that the solutions for proposal categories were not being validated. Hence, an Advisory Board member could have whitelisted a proposal for a given category, that would execute something else in practice. This would imply members are voting on some upgrade but instead may be actually voting on something malicious without their knowledge.

The proposed solutions were:

1. Directly address the bug by adding the missing validation. This would ensure that proposal’s actions are properly restricted by the proposal’s category.
2. Add a generic guard against malicious proposals. This includes a speed bump of 24h for all governance actions and the ability for AB members to cancel potentially malicious proposals during this speed bump.
3. As a precautionary measure, we decided that if a member created a proposal subject to member voting unexpectedly, we would hit the emergency pause.

We started implementing the agreed solutions shortly after, while continuing reviewing other potential implications of the bug.

On Friday, after further investigations, we discovered that some categories of proposals can’t be stopped using the Emergency Pause. To prevent this from being abused and mitigate such governance-related issues in the short term, we took the following measures:

• Enforced whitelisting of all governance proposals by Advisory Board members, as a way to prevent members from raising malicious proposals in the first place.
• Updated governance settings so the Emergency Pause category cannot be passed unilaterally by any individual Advisory Board member or the Owner. With this in place, no category proposals can be passed unilaterally.
• Ensured that all Advisory Board members are using hardware wallets, thus making it harder to carry out an attack using this vulnerability.

While there are no indications that any of the board members are compromised, until the full upgraded solutions are in place, we are effectively assuming the following:

• An attacker would need to compromise at least three Advisory Board members to pass a malicious proposal, subject to Advisory Board voting.
• An attacker would need to compromise an Advisory Board member and compromise member voting to pass a malicious proposal subject to member voting.

For further transparency, the Advisory Board members have the following rights:

• Any Advisory Board member can categorise (whitelist) a proposal.
• At least three Advisory Board members are required to set the default outcome for proposals.
• At least three Advisory Board members are required to change category settings for proposals.
• At least three Advisory Board members are required to initiate the Emergency Pause.
• At least three Advisory Board members are required to change any centralised component (such as quote engine, KYC, capital model) or external feed addresses.

For background, the Owner role was originally in place to update addresses for centralised components if required (such as the quote engine and KYC), but also to update currency feed addresses and trading, where we anticipated changes may be required quickly. The Owner role still technically exists but has effectively been merged with the Advisory Board role as it now has the same privileges after all Owner powers were transitioned to Advisory Board voting.

We acknowledge that there is a high level of trust in the board, but it’s worth mentioning that there is an additional legal protection layer where all Advisory Board members have specific legal obligations to the mutual and it’s members.

The permanent fixes are in the final stages of development and will be put in place in a few days. Until that is completed, we are confident that the conservative governance settings that were enacted in response to the disclosure are robust and sufficient to protect against any malicious proposal.

On Thursday morning, February 20th, just before our community call, we received another report from security researcher samczsun detailing a vulnerability that could have potentially put part of the funds in the mutual at risk. We confirmed the vulnerability shortly after. Within 4 hours of receiving the report, we killswitched the system’s interaction with Uniswap which meant the vulnerability could not be exploited.

The mutual’s funds are currently held in DAI and ETH. Whenever a claim is accepted the mutual must ensure enough funds in the corresponding currency are available to pay our members. If the amount to be paid exceeded a certain threshold, the mutual was exchanging the required amount in several batches, with a time delay in-between. The mutual was relying on Oraclize to trigger a rebalance via Uniswap.

There were two bugs making the attack possible:

• The function handling the Oraclize callback was unprotected — allowing any member, not just oraclize to call it. It was initially designed like this, so the function could be called if Oraclize failed. However, this was a bad architecture decision.
• It was assumed that the function would only be called once per query ID either by Oracalize or a member (calling it after Oracalize failed); thus lacking replay protection

We do acknowledge that there have been warnings against using Uniswap in this way and it is known to be susceptible to this type of attack. The bZx hack should have been a huge red flag for us as well, but we were overwhelmed by our product being put to its first real test to successfully pay a claim. While we should have done this sooner, we are now moving away from this implementation. We took this as a wake up call to review all of our 3rd party integrations and develop emergency plans in case of their failure. This will be documented and shared with the community in the upcoming weeks.

It’s critical for the mutual to have enough reserves in a given currency to honour all accepted claims. In the short term, if claims in DAI are to be paid, we’ll raise a governance proposal to transfer the required ETH to the Advisory Board multi-sig, exchange them for DAI, and pay the claim. Long term, we’re looking to integrate with a manipulation resistant dex, and are currently evaluating options.

Will the disclosures be compensated?

Mudit Gupta will receive a $2000 bounty for their responsible disclosure. Based on the threat matrix, we categorized this as a medium severity issue, due to its high impact and low likelihood of being executed.

Samczsun will receive a $5000 bounty for their responsible disclosure. Based on the threat matrix, we categorized this as a high severity issue, due to its high impact and medium likelihood of being executed.

Additionally, we’re launching our own bug bounty program. More details will follow.

Is our code audited?

Our code has been audited at launch by the Solidified team in April of 2019. We have been pretty conservative when it comes to code updates, and have only ever made three minor upgrades since then:

1. Dynamic Capital changes which added an additional parameter to the Minimum Capital amount.
   Before the change, the minimum capital amount was 7000 ETH.
   After the change, the minimum capital amount is 7000 ETH plus a variable minimum capital that increases each day the MCR% exceeds 130%.
2. Enabled members to switch their address.
3. Replaced the MakerDAO feed with Chainlink for the ETH-DAI price.

Prior to the disclosures, we had scheduled a new full audit set to begin at the end of the month in order to integrate the pooled staking upgrades. This will proceed shortly.

Why haven’t you hit the emergency pause?

While we had the option to hit the emergency pause, we’ve mitigated all known attack vectors. Triggering the emergency pause would also introduce other difficulties for our members, like delaying potential claim payouts and member funds being locked for the duration of the pause. We are working on comprehensive fixes that require contract upgrades and further security reviews.

Lessons Learned

• For a successful responsible disclosure, the team must be reachable at any given time. We have updated our security contact details that will notify us 24/7 if anything is reported.
• Bug bounty programs and responsible disclosure policies are good things to have in place to encourage proper reporting, rather than exploitation, of issues. Although we do not have a lot of funding as a team, we take this seriously and have decided to launch our own bug bounty program. Details on this will follow shortly.
• Having security response procedures in place is critical to allow us to respond promptly and effectively to such events. It is now our policy to have a clear emergency plan that all team members are familiar with. We will ensure mitigation investigation and report confirmation will be launched in parallel, with each sufficiently and individually resourced.
• Code reviews and audits won’t find all the bugs. You can never test your smart contract functions enough. Moreover, Defi applications are susceptible to a wide range of economic attacks. We’re incorporating this invaluable threat knowledge base developed by @sambatcha in our development practices, and looking to expand the security expertise in our team. If you’re a solidity developer passionate about defi and security, please get in touch!

We’d like to thank both Samczsun and Mudit Gupta once again for their disclosures. The whole Ethereum ecosystem benefits from their work and the more we all encourage work like theirs, the safer all user funds will be. And also a big thanks to the entire Nexus team for working hard over the last few days. We have taken these disclosures very seriously, have learnt a lot during the process, and will continue to focus and improve on all security related matters.