Thoughts on $110M Mango Markets Exploit

Austerity Sucks
14 min readOct 16, 2022

--

Disclaimer: I am a MNGO holder, have been a member of the DAO, and involved in the project from very early on. I am biased in favour of Mango Markets because I know and trust the core team members.

I’ve been working on crypto markets product design since 2014 and worked with numerous centralised and decentralised exchanges offering a variety of products varying from spot-margin, futures, perpetual swaps, options, prediction markets. Around 2017 my focus shifted away from centralised exchange products to facilitating these kinds of instruments in a fully decentralised context to maximise the benefits of censorship resistant permissionless characteristics of crypto. After working on layer 2 solutions on Ethereum for doing leveraged futures orderbook trading, I realised Solana was a much better foundation for facilitating a “proper” (non-AMM) market in a central limit orderbook. One of the many projects I’ve invested in and advised in this journey was Mango Markets, which has an incredibly strong team that has built a great product — not just in terms of the trading functionality but also the DAO to decentralise the management of the protocol.

Mango allows users to deposit various assets into the protocol. These assets are automatically earning interest as part of a lending pool that borrowers in the protocol can draw from. On top of that, there is a perpetual futures market, which is a zero sum game, for multiple pairs. This creates an experience where you can do spot-margin trading and futures trading in one account, cross-margined.

This week, someone managed to exploit the protocol to max out the borrows against all the deposits in the platform to take $110 million of equity out, leaving depositors empty-handed.

The exploit required the following:

  • Significant capital: $10M in collateral deposited to multiple Mango accounts, $2–3 million to manipulate underlying spot markets (on- and off-chain)
  • Wash trading: Built up a 480 Million unit MNGO-PERP position between his own accounts (so, wash-trading, not economic fills, would not be fillable in normal economic conditions)
  • Manipulating off-chain KYC-requiring Centralised Exchanges: Unlike many DeFi hacks that are purely on-chain and involve various contract calls / program interactions to exploit funds — this incident involved a willingness to use kyc’d centralised exchanges to manipulate the spot price of those books
  • “Dumb” Oracles: MNGO-PERP’s value was marked at Switchboard v1’s MNGO/USD feed consisting of FTX, Ascendex, and Serum — an estimated $2–3 million of capital was used on these venues to push MNGO price up to $0.40+ (10x higher than $0.038 levelwhere it was…). This oracle was “dumb” in that it only took simple median of these three inputs — with no additional quality controls to resist manipulation.
  • Withdrawing against UPL: That manipulation led to the MNGO-PERP position being in a massive unrealised profit (UPL) of over $100M. That UPL was then used to borrow a variety of assets from the lending pool (consisting of deposits from other users in Mango protocol) and pull out of the Mango protocol

So, in summary: he added 10M USDC collateral to Mango, wash-traded the massive MNGO-PERP position, then used $2–3million to pump MNGO/USD on FTX and Ascendex to manipulate the MNGO/USD oracle price 10x higher, which led to his position to be marked at a humongous unrealised profit of over $150M, which he used as collateral to borrow $110M of all the available assets depositors had in the platform and withdraw it off.

The cash markets recovered shortly after to around the pre-manipulation price, and the mark price dropped, leading to the account being massively underwater and causing a system loss equal to the magnitude of the funds taken. This loss, meant that all available assets deposited in the protocol were taken out, and as such, all depositors funds were now unavailable.

Finally, after dumping millions of MNGO, he decided used a stack of MNGO that was taken in order to make a proposal on the DAO (which carries a requirement of 10M mngo) whereby he was offering to give money back:

While not directly relevant to the execution of the exploit or surrounding risk factors of the protocol, it is relevant in terms of the legal perceptions.

What Protocol Features Allowed This?

This attack was not so sophisticated that it was not possible to mitigate.

There are a number of characteristics of how the Mango protocol works that contributed to the ability to execute this exploit.

Issue #1: Why was a 480 Million Unit MNGO-PERP position able to even be formed? An OI limit should have been in place to limit the damage that could be done on an illiquid market. The circulating supply of MNGO is 1 billion, so theres no reason for letting half of the circulating supply in position. If there was a limit of say 50 Million, 5% of supply, then the payoff would have been more like $11M , and at a cost of $12M it would not have been economical to do the exploit.

Issue #2: Leverage offered in MNGO-PERP too high — was effectively 4x

If the leverage was much lower, like 1x, on MNGO-PERP, then it would have required more collateral to conduct the attack, lowering the effective yield to a point where it would not be economical to conduct it.

Issue #3: Risky treatment of UPL when validating withdrawals

The attack relies on UPL being treated free and clear as collateral to support borrowing and pulling assets off the protocol — if the withdraw function instead weighed UPL as 0 it would not have allowed the attack.

Issue #4: More robust index price using more components

It is up for debate whether it is the job of an Oracle / Index provider to be cleaning the data produced from underlying price sources, or whether that is the job of the consumer.

In any case, three constituents is not ideal, and even if just being a “dumb” oracle price simply communicating latest price, there should be efforts to make strong, dependable index feed.

So, allowing a constituent like Ascendex which has very poor liquidity shows a bad oracle design. A proper oracle should be monitoring the quality of a given market and having exclusion criteria to protect against including a venue that may be susceptible to manipulation.

Issue #5: No separation of index price and mark price as a concept

MNGO was just manipulated 10x in one minute — if the index was “stupid” to just report blindly the recent trades or recent mid price without any cleansing or quality assurance, then it’s on the Exchange / venue to be adding these additional constraints. Mango could have consumed the Index, and then added controls for how the asset’s value would be marked so that it would limit how fast it could go move in a given time period — say to no more than 3x in a given hour. This would have given the spot-market time to adjust to the manipulation and then the rise would not have been enough to make the exploit work.

Was This Risk Known?

The general attack vector of someone manipulating components of an index or manipulating the oracle itself, in order to value collateral or positions in an off-market way to pull funds off a platform, is a well-known issue in risk-systems. Whether it’s a simple borrowing/lending protocol or a derivatives protocol — you always need to be wary of market manipulation that can lead to losses for those providing the leverage (depositors/lenders).

That said, the degree to which this is exploitable is dependent upon the quality of the assets supported as collateral or for perpetual positions. Once you start adding lower and lower quality assets (not just BTC, ETH, SOL where theres a robust global market of dozens of venues supporting liquidity), you need to then be very careful about key risk controls:

  • Leverage limits
  • Collateral weight
  • Position limits (not as effective in a frictionless on-chain context where its trivial to spin up multiple accounts and spread)
  • Open Interest Limits
  • Borrow limits
  • Stronger Mark Price Controls

Just to name some — and all of these levers allow for providing some amount of leverage on lower quality assets, but with heavily limited risk.

So this was not really an issue when Mango v3 was launched — until the protocol was starting to add lower-quality assets, and it really started around September 2021, when there was populist desire in the DAO and Solana community in general to support trash tokens that had low market quality.

This was very controversial and there were a lot of discussions on Mango dev meeting on Zoom, on Discord, the Mango Forum, and on Twitter:

I made a post on the Mango Forums to this effect https://forum.mango.markets/t/on-the-remaining-9-oracle-slots-prioritising-perp-listings/129/1

Highlighting all the issues related to asset quality and the different controls needed to avoid serious consequences:

Basically, an asset’s “quality” — covering how volatile it is, how liquid its underlying markets are, the supply characteristics, etc. — should dictate how much risk the users and the system are willing to take in terms of leverage and exposure size. Failure to control these with proper parameters — and keeping those parameters up to date with the latest relevant data — can lead to potential catastrophic losses for the system and users.

The counterpoints were about Mango needing to have a “competitive advantage” in offering more exotic products, and that Mango needs to take such risk. My retort to this was, that if we wanted to go in this direction, that the risk should be segregated, to different depositors / traders who are willing to take that materially higher risk. However, supporters of the COPE, STEP, SBR and other projects effectively raided the Mango Discord to present an appearance of popular support. I made detailed responses to make the case that these assets are too low quality and should not be supported with any margin, such as this one regarding STEP:

Some battles I won, some I lost. In DAO driven by popular democracy style principles, the apes end up prevailing more often than they should.

Ultimately, COPE was voted in and approved by the masses, signaling the beginning of a shift for the DAO towards a willingness to allow for more risk.

And about 9 months later, in July 2022, there was manipulation using COPE:

Some actors were executing baby versions of a variation of the attack — where COPE was being pumped on FTX to impact the market value of COPE holdings, in order to pull USDC out of the system — on the order of $500K taken. It did not directly involve a PERP, but the core mechanisms of the exploit were the same: build a position on Mango, pump underlying cash markets to manipulate oracle, and take out a loan against it to pull off platform. This is about 2.5 months before the MNGO-PERP Avi exploit. Liquidators shouldered the burden of the losses there.

This led to delisting COPE support:

So, the risk got cleaned up there and there was still some issues in the system, namely MNGO-PERP, where this could happen again.

This was one way to handle the risk — was to delist the asset and perp from the platform. Another way would be to add the risk limits described above. The problem is that the way v3 was built it was not trivial to introduce the types of limits needed to sufficiently mitigate the risk.

And so the focus was more to fix these things in new version of the protocol, v4, where it could be cleaned up properly. In the meantime, the risk was there on v3, for MNGO-PERP, but only if the position was of sufficient size, which carried with it a collateral requirement of over $10M. It also required a willingness to spend millions manipulating the MNGO price on different venues, taking the risk on a kyc centralised exchange, not just on-chain. So, taking this into account, the urgency of this risk was not such that the whole protocol would need to be shut down to prevent it, due to the significant barriers to doing this exploit. It was not like it was just a simple contract call that anyone could do — it required shitloads of capital, and a lot of other “real-world” (off-chain) activity to bring it all together.

In hindsight, when winding COPE down, MNGO should have also been wound down, and even many other perps.

Trade? Exploit? Hack? Legality and Ethics

The guy who did the exploit, Avi Eisenberg, is of the opinion that this was simply a trade and he used the protocol as intended:

“a highly profitable trading strategy”…

I think it is a bit more complicated than just saying the protocol was used as intended. It required more than just interacting with the smart contract on Solana. It required off-chain actions — coordinating pumps on centralised exchanges (FTX, Ascendex) in order to impact the Switchboard oracle and then the Mango UPL to then drain all the depositors. So, it was not just some clever arbitrage in an on-chain program, using flashloans to orchestrate a series of on-chain interactions. However, it’s also true that the protocol was not “hacked” — there was no unauthorised interactions with the contract, he did not steal anyone’s private keys to gain access to things he otherwise should not. In that sense it’s not a hack IMO either.

Instead I would call this just an “exploit”. It should be pretty obvious that the intention of the protocol is not to allow somebody to take a position against themselves in order to take all the depositors money. So by using the protocol in this way it’s exploiting the design and the reliance on illiquid KYC’d centralised exchanges to be able to take a bunch of money.

Regarding the legality of this — I do not know enough about the law to say. I do not think that he should be criminally charged per se — but I also do not think he should have taken a $45 million pay-day on a $110 million sploit — the 10% rule is the market rate — so $10M is the fair value where gentlemen of crypto can agree “okay, this was critical, you had the means to demonstrate this was exploitable and did it, you have earned this bounty”. But $45 million is greedy.

Also, he is a US person residing in the US (allegedly, Puerto Rico), so the only way he would have been able to access Mango is to misrepresent his location with a VPN or something:

I would also say that this is clearly unethical behaviour — there are clear damages being inflicted based on non-economic trading activity solely for the purpose of taking depositors money. You could say, well, when you deposit to Mango your funds are going into a lending pool that is inherently at risk and so people should know that they may potentially lose. But, it all comes down to intent for me — the intent here was clearly to exploit and take the money out of the protocol. It was not some loss that happened from people who were just intending to use the protocol normally to trade and borrow.

Code is Law

Yes, code is law — the smart contract interactions get processed if they don’t violate the network rules. The code you write and use on chain will execute in a predictable and reliable way.

However, it does not mean that any interactions with smart contracts are inherently virtuous or ethical. If something is exploitable, it will be exploited — and in crypto we absolutely need to be building systems that are robust to the most adversarial actors, and we should NEVER be relying on good will or “nice guys” to be doing the right thing. The code and the economic incentives need to align perfectly to ensure that the system is robust.

But this doesn’t mean that if somebody is combining a bunch of off-chain activity to induce on-chain activity that facilitates an exploitative use of a protocol — that this is justifiable, and morally acceptable. If the intent is clearly to steal, then it is unethical.

A Highly Profitable Trading Strategy

A different “highly profitable trading strategy” one can consider is the payoff of pursuing legal action against Avi:

  • Cost of investment: lawyer costs ($2–3 million?), time
  • Possible payoff: $45 million representing the remaining funds
  • EV dependent upon probability of winning lawsuit and probability that the funds are recoverable in the event of a win

Is this +EV? -EV?

Now, bear in mind, I’m not suggesting anyone sue him — I am simply thinking through a trading strategy here.

Let’s assume that there’s a 50% chance of success in the civil law suit — it’s hard to say but a lot of commentators are referring to the vagueness of fraud statutes and suggesting that this would be a successful case for any MNGO holders who have been damaged and maybe even depositors who took the loss (despite it being repaid).

In fact, Eisenberg is suing Waves and claiming that they were doing price manipulation and conspiracy to defraud him when he tried to do a similar trade and failed!

So, lets just say theres an 50% chance it will work out.

But even if it succeeds, maybe Avi decides to lose the money and refuse to pay? Maybe he gets separate criminal charges and the feds seize the money and its not retrievable? Lets attach a 20% chance to that:

  • (45 million * 0.50 * 0.80)-3 million = $15M

Maybe it’s +EV after all to do this trade…

FAQ

Since I have talked about the project in the past lots of ppl asking me random things so I will add here:

Q: Was it an inside job?

A: I don’t think so. But it’s a possibility — at least, it’s not a 0% probability

Q: How much did you lose?

A: Obviously not going to say, but I had decent size deposit in Mango and have a fair number of MNGO tokens. The deposit value at least looks safe, but MNGO has been an -80% drawdown from the IDO price I invested in.

Q: Are these guys amateurs or how did they let this happen?

A: As I tried to explain in the main post, there was not a lack of understanding of the financial/risk concepts. Discussions going back to pre-v3 were had involving many DAO members which covered all the key considerations. There was not really something where you think “omg, I can’t believe we did not think of that”, but when you are building and shipping stuff you have to make calls on weighing the effective risk of one decision vs another. I think a lot of the people in the DAO were willing to take more risk than were completely understood, as indicated by COPE being added. But the risk was understood, which is why there were conversations about splitting the pools between high and low quality assets.

--

--

Austerity Sucks

aka swapman. I'm co-admin of Whalepool.io and do stuff with cryptocurrency derivatives.