Introducing Harbinger: a Self-Sustaining Price Oracle for Tezos
By Luke Youngblood and Keefer Taylor
Harbinger (noun): Something that foreshadows a future event; something that gives an anticipatory sign of what is to come
Today we’re excited to present Harbinger, a project to deliver signed price feeds based on market data from multiple exchanges to the Tezos network — and, we hope, a sign of what is to come.
How does the Harbinger Price Oracle fit into Tezos?
Tezos is a liquid proof of stake cryptocurrency network, so account holders can delegate their Tezos to a validator (referred to as a “baker”). Delegators receive a share of the block rewards they earn by helping to secure the Tezos network. In Harbinger, an account that pays for fees to update the price oracle can be delegated and pre-funded with tez. This enables the development of self-sustaining price oracles, where the rewards for participating in proof of stake consensus offset the fees required to keep the oracle data current.
Below we dive into oracles and how they’re a key building block of Decentralized Finance or “DeFi”. We’ll then provide an overview of the Harbinger Price Oracle and why it marks an important step towards creating a Decentralized Finance ecosystem on Tezos.
If you just want to get started with the free, open source Harbinger software right away, click here to jump to the ‘Getting Started’ section.
Establishing Trust in DeFi
What is Decentralized Finance (DeFi)? Essentially, it’s an ecosystem of financial applications that are open source and built on top of blockchain protocols. It’s decentralized because it’s a system that moves away from traditional, centralized financial services and all the intermediaries attached to them.
Over the last two years, we have seen DeFi rapidly grow into a core use-case in crypto. As of mid-August 2020, there are $6.73B in assets actively pooled in DeFi protocols and applications that offer a wide variety of useful financial services, including stablecoins, borrowing and lending, decentralized exchanges, derivatives, and options.
In order for these DeFi applications to run properly, they need an “oracle” to provide a trusted price feed. Think of oracles as tunnels that link on-chain data to off-chain data, providing protocols and applications the external information that they need to function properly. Financial price data is critical to DeFi, allowing protocols to build algorithmic stablecoins, write derivatives or futures contracts, create collateralized loans, and build insurance products.
Despite the essential function oracles play connecting blockchains to external data, there is the ever lingering question of trust. When so much value depends upon accurate price data, how can we trust the information oracles have relayed in? How do we know the data hasn’t been manipulated by bad actors looking to take advantage of inaccurate pricing?
A relatively new development in DeFi are exchanges that create cryptographically signed price feeds, such as Coinbase Pro and OKEx. This allows anyone (which we call a “poster”) to take a signed price from one of these exchanges and post it on-chain. Prices are signed by the exchange’s private key. Proper storage contracts, once deployed, will only accept prices that were signed by the proper key and will therefore be trustworthy, regardless of the poster.
Introducing Harbinger: The Trustworthy Oracle
Harbinger is a set of tools and reference contracts that allows anyone to become a poster. As a poster, they can deploy a price oracle on the Tezos network that will publish cryptographically signed prices. Additionally, Harbinger includes a reference signer that an exchange, at a low cost, can deploy in order to provide a signed price feed compatible with Harbinger.
“Having a reliable feed for on-chain price data is critical for DeFi lending platforms. Harbinger is an important building block for the decentralized finance ecosystem on Tezos.” — Robert Leshner, Founder of Compound
Harbinger was inspired by Compound’s Open Price Feed, but adds some additional features. Essentially, the process works as follows: first, the signer retrieves market data from an exchange and signs it with a private key. A poster can then retrieve prices from the signer and post them to the storage contract. Optionally, a poster can invoke a callback on the storage contract that will push the price data to a normalizer contract. The normalizer contract calculates a volume weighted average price, which gives price data consumers assurance that the normalized price won’t be influenced by low-volume price outliers. Finally, any DeFi protocol may invoke a callback on the normalizer contract in order to have the latest normalized price pushed to their own contract.
If you have questions regarding this process, don’t worry, we’ll take a deeper dive into the various components of Harbinger in the next section. However, if you’re just interested in building, click here to jump to the ‘Getting Started’ section.
The Signer
Harbinger provides a reference signer that supports the market data APIs of Coinbase Pro, Binance, Gemini and OKEx. It contains an extensible architecture to enable other exchanges to rapidly provide the same signed price data. Harbinger’s ability to provide price data from multiple sources will make DeFi protocols more trustworthy and secure. Protocols and applications will have the necessary information to calculate a price anchor that is medianized across data providers. This process will dramatically reduce the likelihood of inaccurate data.
The Harbinger reference signer is a Serverless Framework application that can be deployed at low cost on Amazon Web Services. It leverages Amazon Key Management Service (KMS), which uses hardware security modules as the root of trust, storing private key material and signing price data. It is impossible for anyone to extract the private key material when KMS is used in this way. As with price feeds, key management easily extends to any data provider’s key management needs.
Besides increased security, there are other perks the Harbinger reference signer brings to the table. Its deployment can be completely outside an exchange’s internal network. Not only is there no need to maintain or operate servers, but it can also generate signed prices for a configurable list of market order books. Additionally, the signer includes a secure caching API gateway which enables posters to retrieve prices without generating more than 1 request per minute to market data APIs per order book.
The Poster
Harbinger price oracles will be reliable and keep price data constantly updated. They will do this by giving posters the opportunity to retrieve signed prices from either a signer or exchange, and post these prices on-chain whenever they wish. The Harbinger price storage contract, which we will take a closer look at in the following section, will only accept price data that is signed by a given signer’s private key. This ensures decentralization in the updating prices process, where the failure of a single poster will not result in stale prices.
Harbinger posters can function in two ways: as a Serverless Framework application, or alternatively, they can run a command line utility as a background process that continuously retrieves prices from a signer and posts them on whatever interval the posters prefer.
The Harbinger Contracts
Harbinger provides reference implementations of the price storage and normalizer contracts. Harbinger users can provide custom implementations which optimize for gas, storage, or data points while still leveraging the rest of the Harbinger system.
The Price Storage Contract
The Harbinger price storage contract is a reference contract written in SmartPy and compiled to Michelson, the smart contracting language of Tezos. It supports a configurable set of market order books and a public key of a data provider. It stores the latest price candle (a candle includes the start/end time, open, close, low, high, and volume) that was retrieved from a specific signer. In the unlikely event that a signer is compromised, the storage contract supports a revoke entrypoint, allowing a signer to signal to the network that the signed prices should no longer be trusted.
The Normalizer Contract
The Harbinger normalizer contract is another reference contract that supports normalization of price data across multiple updates by employing a volume-weighted average. You could choose to modify the contract to perform other math, such as an exponential weighted moving average. You can also configure how many data points you would like to normalize.
After posting a signed price, a poster has the option to invoke a callback to the storage contract, which will send the same price data to a Harbinger normalizer contract. These two operations can be performed as a single atomic transaction on the Tezos network, meaning posters can post the latest price data and update the normalizer contract of their choice.
DeFi Protocols and dApps
DeFi protocols and decentralized applications (dApps) can use price data from the storage or normalizer contracts by simply invoking a callback on either, allowing data to be pushed to their contracts. In order to avoid reentrancy attacks, Tezos smart contracts prefer to use callbacks to receive data in this way.
Harbinger CLI
The Harbinger command-line interface enables you to easily deploy the above mentioned contracts and keep them continuously updated, introspecting the market data in those contracts.
Getting Started Using Harbinger
This section is for builders who would like to get started using Harbinger.
The Tezos community has already deployed initial versions of these contracts on CarthageNet and Mainnet that you can access below. Please note that the contracts deployed will only accept updates that are signed by the private keys held by Coinbase Pro (for the Coinbase contracts) and the community signers deployed by Blockscale (for the other contracts).
Carthagenet
Coinbase
- Coinbase Pro price storage contract: KT1P7D7jt3PfjMpsEKPyao1kHQR93t7XR5zh
- Coinbase Pro normalizer contract: KT1VsWxgE683MiXoaevLbXtpJqpWrcaWaQV7
Binance
- Binance price storage contract: KT1XZitEQjpKRUC1k5gpW5qVehiKqB7BfuJn
- Binance normalizer contract: KT1Jo1txCBvGvpr3z11btiNvuFT7XG3dGkqy
Gemini
- Gemini price storage contract: KT1VZprQjiuHG9qz2F7hhiSgzttq9qw6Vvb9
- Gemini normalizer contract: KT1N897aHVKtZ2UTbhw6DydZW7aRCUYff4E9
OKEx
- OKEx price storage contract: KT1XnxcyYsPGXb5pCFeykcQvXggTcY1qfX1i
- OKEx normalizer contract: KT1KS2HGyYuoVuNgqGh5947DhmZu7JsAyy58
Mainnet
Coinbase
- Coinbase Pro price storage contract: KT1Jr5t9UvGiqkvvsuUbPJHaYx24NzdUwNW9
- Coinbase Pro normalizer contract: KT1AdbYiPYb5hDuEuVrfxmFehtnBCXv4Np7r
Binance
- Binance price storage contract: KT1Mx5sFU4BZqnAaJRpMzqaPbd2qMCFmcqea
- Binance normalizer contract: KT1SpD9Xh3PcmBGwbZPhVmHUM8shTwYhQFBa
Gemini
- Gemini price storage contract: KT1Jud6STRGZs6hSfgZsaeztbkzfwC3JswJP
- Gemini normalizer contract: KT1JywdJbaVW5HtsYh4XNNuHcVL2vE6sYh7W
OKEx
- OKEx price storage contract: KT1G3UMEkhxso5cdx2fvoJRJu5nUjBWKMrET
- OKEx normalizer contract: KT1J623FNZ6an8NHkWFbtvm5bKXgFzhBc5Zf
Reference Signers
- Coinbase Pro (signed by Coinbase Pro) — public key: sppk7bkCcE4TZwnqqR7JAKJBybJ2XTCbKZu11ESTvvZQYtv3HGiiffN
- Binance (signed by Blockscale) — public key: sppk7cJiWJyVQ9uURPRTD9hiKqc93aJ1Br4o5G5QWZjJV9LBAoS6jkd
- Gemini (signed by Blockscale) — public key: sppk7cJiWJyVQ9uURPRTD9hiKqc93aJ1Br4o5G5QWZjJV9LBAoS6jkd
- OKEx (signed by Blockscale) — public key: sppk7cJiWJyVQ9uURPRTD9hiKqc93aJ1Br4o5G5QWZjJV9LBAoS6jkd
Post/Publish or Introspect Price Data
- Install the Harbinger CLI or deploy the Serverless poster.
Contribute to the Ecosystem
- Use Harbinger-lib to build components and programs.
Deploy Harbinger CLI, Signer, Poster and Contracts
All of the tooling in the Harbinger ecosystem is fully open-source and has been released under the MIT License. If you’d like to use the tooling to deploy, sign, or post to your own contracts, you can find the main repository with complete documentation here.
Conclusion
The Harbinger price oracle ecosystem is just getting started, and we can’t wait to see what new and interesting DeFi protocols and applications are built with it. Algorithmic stablecoins, lending platforms, and insurance products might be some of the first use cases for these new oracles.
If you’re building anything interesting on Harbinger, we’d love to hear from you. Please reach out to Luke Youngblood or Keefer Taylor directly.
Acknowledgements
This was a fun project, yet we would not have been able to build it without standing on the shoulders of giants who helped us along the way. We’d like to thank the following individuals who helped us with technical guidance, feedback, and encouragement:
- Jacob Arluck and Michael Klein from the Tocqueville Group
- The SmartPy Team for their SmartPy smart contract language and vibrant Telegram channel
- Mike Radin from Cryptonomic who handled our feature requests and provided integration assistance
- Robert Leshner and the entire Compound Finance team
- Arthur Breitman
Who we are
Blockscale is a security-focused engineering company that designed and operates proof of stake validators for the Tezos Foundation and other key proof of stake networks. We pioneered the first known remote signing implementation using HSMs for proof of stake validators in early 2018, and have been active open source contributors in the Tezos ecosystem. We’re interested in exploring the boundaries of decentralized finance, proof of stake, governance systems, and decentralized autonomous organizations (DAOs).
