Harbor Protocol: Oracle Incident Report

Dear Community,

​​On June 29, 2023, stATOM vaults on Harbor Protocol were liquidated due to an oracle error, leading to inaccurate prices of stATOM and CMST. Below is a detailed incident report that outlines what caused the error, along with some actions that will be taken to prevent such occurrences in the future and how we intend to help impacted users.

The Oracle Error

At Harbor Protocol, we utilise a custom script through Band Protocol to fetch asset prices from three distinct data sources in the case of stAtom — Coingecko, Osmosis, and cSwap. On 29th June 2023, a sudden surge in trading volume on ShadeSwap was observed, caused by its new listing on Coingecko, which led to a temporary deviation in the stAtom prices. The price of stATOM was inaccurately reported as ~$0.02, instead of the actual ~$10, on ShadeSwap.

Coingecko, from what we understand, employs a volume-weighted price mechanism and registered this aberrantly low price as the actual value due to ShadeSwap’s inflated trading volume. This discrepancy was displayed on Coingecko for approximately 40 minutes (from 2:20 PM — 3:00 PM UTC). The unusual pricing incident on Coingecko critically impacted our price averaging. While Osmosis and cSwap reported the correct stAtom price at ~$10, the flawed data from Coingecko resulted in a skewed mean price calculation of ~$6.67 ((10+10+0.02)/3).

This led to the liquidation of stAtom vaults at $6.67, triggering auction at a 15% buffer — starting at ~$7.67, significantly discounted from the market rate of $10. Such situations are particularly challenging when dealing with liquid staked assets, which are yet to build depth of market liquidity. Majority of LST volumes in the Cosmos ecosystem are currently limited to DEXes. From observing the nature of the incident, other assets such as stkATOM, CMST and others are likely to also have been impacted by this anomaly.

It is worth noting that this issue has had an impact on stATOM vaults ONLY. Please note that the values presented in this blog are approximations and do not represent actual prices.

Oracle on Harbor Protocol

On Harbor, a custom script fetches the price data from our three data sources (Coingecko, Osmosis and cSwap) every 20 blocks (~2 minutes). The real-time data is stored directly on-chain in our market module, ensuring transparency and accessibility.

To further refine our price accuracy, we employ a time-weighted average currently set at 30, giving us an approximate time frame of 1 hour for our computations. The combined average of these price points is then utilised as the primary reference price on Harbor Protocol.

For further insights into our calculations, you can check our data at https://rest.comdex.one/comdex/market/v1beta1/timeWeightedAverage.

We also maintain open-source Oracle Scripts and Data Sources for community review and contribution. These can be accessed on our GitHub repository: https://github.com/comdex-official/Oracle-scripts.

Our deployed Oracle scripts and data sources can be viewed at https://www.cosmoscan.io/oracle-script/42 and https://www.cosmoscan.io/data-source/100.

Corrective Actions

Oracles are a fundamental pillar to keeping DeFi ecosystems alive. It is therefore imperative to ensure continuous improvements in oracle mechanisms for user and protocol safety. On an immediate basis, we have modified our existing oracle script to remove Coingecko as a source as it continues to display inconsistencies in data for multiple assets (beyond stATOM alone).

We will seek to increase the number of sources from where we aggregate prices going forward as well as adding extra checks on oracle logic to tackle scenarios where significant deviations in prices from a single source lead to a reduction in their weightage.

We have also been in touch with the Ojo Network team for integration of their oracle for having a wider range of decentralised oracle options. We will continue to monitor and improve the protocol to ensure user safety.

We appreciate your patience and continued support!