Should You Consult an Oracle Before Building a Decentralized Application?

Published in

Sipios

13 min readSep 2, 2021

Among all the manifestations of ancient Greek religion, the oracular art - or, mantic revelation - is one of the most loaded with significance. The oracles were considered as portals through which the gods spoke directly to men.

The most important oracles in ancient Greece were the Pythia (priestess of Apollo at Delphi), and the oracle of Dione and Zeus at Dodona in Epirus. These oracles often provided information to the protagonists and travelers during consultations for critical decisions.

By design, blockchains, like the mortals depicted in ancient Greek classics, don’t have a way to access information outside their network. Oracles aim to allow smart contracts to interact with data from the outside world.

Let’s slow down and define few concepts from scratch!

If you’re already familiar with the blockchain basics and Ethereum network, you can go straight to the main purpose of this article: Oracles. (~8 minutes read).

If you are new to this ecosystem, stay with us! You will find the necessary definitions required to understand the concepts behind this article in the following section (~4 minutes read).

What is a blockchain?

A blockchain is a public database that is updated and shared across many computers in a network.
A “Block” refers to the fact that data and state are stored in sequential batches or “blocks”. If you send an ETH to someone else, the transaction data needs to be added to a block for it to be successful.
“Chain” refers to the fact that each block cryptographically references its parent. A block’s data cannot be changed without changing all subsequent blocks, which would require the consensus of the entire network.
Each new block and the chain as a whole must be agreed upon by every computer in the network. These computers are known as “nodes”. This is so everyone has the same data. To accomplish this distributed agreement, blockchains need a consensus mechanism.

A quick overview of Ethereum

What is Ethereum?

Ethereum is a technology that lets you send cryptocurrency to anyone for a small fee. It also powers applications that everyone can use and no one can take down.
It’s the most widely used programmable blockchain! Others exist such as Solana, Cardano, Tron, NEO…
Ethereum builds on Bitcoin’s innovation, with some big differences. Both let you use digital money without an intermediary such as payment providers or banks. But Ethereum is programmable, so you can also use it for lots of different digital assets — even Bitcoin! Ethereum is for more than payments. It’s a marketplace of financial services, games and apps that can’t steal your data or censor you.
So step into the bazaar and give it a try…

What is Ether (ETH)?

ETH is the lifeblood of Ethereum. When you send ETH or use an Ethereum application, you’ll pay a small fee in ETH to use the Ethereum network. This fee is an incentive for a ‘miner’ to process and verify what you’re trying to do.
‘Miners’ are like the record-keepers of Ethereum — they check and prove that no one is cheating. Miners who do this work are also rewarded with small amounts of newly-issued ETH.
The amount of ether paid is a function of the length of the computation. This also prevents malicious participants from intentionally clogging the network by requesting the execution of infinite loops or resource-intensive scripts, as these actors will be continually charged.

Introduction to Decentralized Application

What is a Decentralized Application?

A decentralized application (dApp) is an application built on a decentralized network that combines backend code (smart contract) running on a decentralized peer-to-peer network and a frontend user interface. Contrast this with an app where the backend code is running on centralized servers.
A dApp can have frontend code and user interfaces written in any language (just like an app) that can make calls to its backend. Furthermore, its frontend can be hosted on decentralized storage such as IPFS.
Decentralized means they are independent, and no one can control them as a group.
Deterministic i.e., they perform the same function irrespective of the environment they are executed.
Turing complete, which means given the required resources, the dapp can perform any action.
Isolated, which means they are executed in a virtual environment known as Ethereum Virtual Machine so that if the smart contract happens to have a bug, it won’t hamper the normal functioning of the blockchain network.

What are smart contracts?

A smart contract is code that lives on the Ethereum blockchain and runs exactly as programmed. Once they are deployed on the network you can’t change them. Dapps can be decentralized because they are controlled by the logic written into the contract, not an individual or company. This also means you need to design your contracts very carefully and test them thoroughly.
Note: in Ethereum, smart-contracts are accessible and transparent — like open APIs — so your dApp can even include a smart contract that someone else has written.

What are the benefits of dApp development?

Zero downtime — once the smart contract at the core of an app is deployed and on the blockchain, the network as a whole will always be able to serve clients looking to interact with the contract. Malicious actors therefore cannot launch denial-of-service attacks targeted towards individual dapps.
Privacy — you don’t need to provide real-world identity to deploy or interact with a dapp.
Resistance to censorship — no single entity on the network can block users from submitting transactions, deploying dapps, or reading data from the blockchain.
Complete data integrity — data stored on the blockchain is immutable and indisputable, thanks to cryptographic primitives. Malicious actors cannot forge transactions or other data that has already been made public.
Trustless computation/verifiable behavior — smart contracts can be analyzed and are guaranteed to execute in predictable ways, without the need to trust a central authority. This is not true in traditional models; for example, when we use online banking systems, we have to trust that financial institutions will not misuse our financial data, tamper with records, or get hacked.

We have come to the end of this quick introduction to blockchain basics and the Ethereum network. If you want to learn more about these concepts and discover new ones, please refer to the resources at the end of the article.

Congrats! You are now ready to consult the Oracle.

In this article, we’re going to walk through the principles of an Oracle, the problems associated with the use of Oracles, and why you may need them. Now, these are crucial pieces to understand because they unlock the full potential of smart contracts. So let’s dive right in!

Why are Oracles needed?

When a transaction occurs in the blockchain, it needs to be validated by every other node in the blockchain so that they can all agree. For example, we can see that when we send five dollars from Bob to Alice, we can simply subtract five from Bob’s wallet and add five to Alice’s wallet.

This transaction can be easily replicated by every other node on the network.

But what if it were not the case? Let’s imagine a blockchain that permits non-determinism and allows nodes to validate each other, based on a variable. Let’s say we want to send a variable amount from Bob to Alice. For example, we want Bob to send the value (in $) of an Ether to Alice, at the time the transaction is created. At which point, the node that Bob is interacting with calls an API to get the value of an Ether and sends it on to Alice.

Now, when all the other nodes go to validate this transaction, they must also call this same API. But what happens now?

Even a moment later, the API could be changed, updated, hacked, or simply deprecated. Consequently, all other nodes will get a different result and won’t be able to validate the transaction.

This means that none of the nodes would be able to actually agree upon what the state of the blockchain is.

That’s why Ethereum blockchain and blockchain, in general, are designed to be entirely deterministic. This means that if we replay each transaction from the genesis block (the first initial block), we should end up in the same state. If we add API calls to these transactions, we will end up in wildly different states and no node will be able to have a consensus with what the actual state of the blockchain is.

Now, at the same time, we need blockchain and we need smart contracts to be able to connect to the outside world and connect to APIs. We need, for example, to get the price of financial assets so that we can create financial products and services accessible to anyone (DeFi) — anyone with an internet connection. Another example might be that we need to get meteorological data to have decentralized agricultural insurance contracts. Access to real-time data is essential to use blockchain to its full potential. These are some practical examples of the uses of oracles.

What is an Oracle?

In the blockchain realm, an oracle is a one-way digital agent that seeks and validates real-world facts before cryptographically submitting them to the inquiring smart contract.

“An oracle is not the data source itself but the layer that interfaces with data-sources & the blockchain; it’s a translator for information provided by a 3rd-party API that’s to be added to a blockchain.” — J. Najera

Smart contracts have a way to interact with data outside of the blockchain context thanks to oracles.

How do we bridge the world with this constraint?

Existing oracles, such as Chainlink, WINkLink, or iExec RLC, require asynchronous* interactions between smart contracts and data layers to achieve information.

*Asynchronous — a programming paradigm that allows a unit of work to run independently from the main flow. When the work is finished, the main flow is notified and the execution is resumed (or stop if the work failed).

Steps using existing oracle solutions:

Contract saves the state of the current transaction to the contract’s storage.
Contract emits an event to request data query and stops the current transaction.
Off-chain (outside of the blockchain) network waits for sufficient transaction confirmation.
Off-chain network invokes a callback transaction with the supplied query result.
Contract validates the transaction, recovers the state, and continues execution.

Two separate transactions are required to achieve information

The off-chain oracle node is responsible for listening for events emitted by its corresponding on-chain smart contract. Once it detects an OracleRequest event, it uses the data emitted to perform a job.

The most common job type for a Node is to make a request to an API, retrieve some data from it, parse the response, convert the result into blockchain compatible data, then submit it in a transaction back to the oracle contract, using, for example, the fulfillOracleRequest function.

Even if this architecture is the most widespread, especially among the largest projects (Chainlink, WINkLink, or iExec RLC), other projects such as Band Protocol propose interesting alternatives. If you’re interested in their solution, you can consult their white paper.

The Oracle problem

Let’s take a step back and remember why we are building on blockchain in the first place. While blockchain technologies often use decentralized networks, a blockchain application itself cannot be categorized simply as decentralized or not.

By decentralizing the management and access to an application’s resources, greater and more equitable service can be achieved. Decentralization usually comes with some trade-offs, such as lower transaction throughput, but ideally, these trade-offs are worth the stability and improved service levels they produce.

However, the data has to come from somewhere. If we import data onto a decentralized blockchain through a centralized or single API source, we remove one of the main benefits of blockchain. In this case, we centralize the logic of our application. Indeed, a single Oracle or a single data source now controls the whole execution of your smart contract.

The Oracle problem can be summed up as these two pieces combined:

Blockchains themselves can’t access external data.
Centralized Oracles nullifies the decentralization of smart contracts.

A solution: Decentralized Oracle Networks

“Invincibility lies in the defence” — Sun Tzu

While the centralized oracle operator may operate with the best intentions, they are still subject to all the common centralized problems of today like downtime, DDOS attacks, hacks, and accidental incompetence, all of which put users’ funds at risk.

Even the noblest centralized entities can come under pressure once the value of the contract scales, opening them up to bribes, intimidation, and regulatory pressure, which only require one person involved in the operation to go rogue. This model is not scalable and doesn’t fit with the idea of decentralized infrastructure being a key driver of secure and reliable automation.

To overcome these shortcomings, oracles need to create the same security and reliability guarantees of a blockchain, although differently given their many differences.

To ensure determinism to the oracle layer, the projects presented above developed a network of decentralized oracle networks (DONs).

For example, let’s say Bob has booked a flight to go on vacation and he buys insurance (via a smart contract) to protect himself in case his flight is canceled or delayed (by an hour or more). The amount of the purchased contract is transferred to the insurer.

If the plane takes off on time, Bob knows he is not entitled to a refund:

If he chooses not to execute the contract. He loses the amount purchased for this insurance.
If he chooses to execute the contract. He loses the amount purchased for this insurance as well as a fee for performing the contract.

If the plane is canceled or delayed, Bob knows he is entitled to a refund:

If he chooses not to execute the contract. He loses the amount paid for the insurance and the amount of the flight.
If he chooses to execute the contract. He loses the amount purchased for this insurance as well as a fee for performing the contract but he will be reimbursed the full price of his flight.

Every time we execute the contract, an oracle is used to know if the plane was delayed or not.

We thus observe that in this case if the API used or the data source is corrupted, deprecated, or hacked, Bob will never be refunded.

This is a big part of why we use decentralized networks of Oracles. Each oracle in the network will be in charge of querying a data source. Multiple oracles need to be aggregated to obtain more accurate results and filter out outliers.

For instance, to create a decentralized application that displays the current value of an ETH in USD, you could use ChainLink Oracle’s network that aggregates the price of ETH/USD from 31 nodes (generates an average or a median price from 31 data sources) and supply the result on-chain. Each oracle operator is rewarded for publishing price data.

In order for an update to take place, the contract must receive responses from a minimum number of oracles. For example, 21/ 31 oracles. Otherwise, the latest answer will not be updated.

ETH/USD price according to Chainlink Oracle Network

About Security

Each DON involves a combination of multiple security techniques:

Open-source — being an open-source technology allows the wider blockchain community to independently verify the security and reliability of source code and functions, as well as contribute to its improvement.
External Adapters — allowing nodes to securely store API keys and manage account logins enables smart contracts to retrieve data from any external system and API, including those that are password/credential protected.
Decentralization — employing decentralization at the node and data source level ensures no one node or data source is a single point of failure, providing users strong guarantees that data will be available, delivered on time, and resistant to manipulation.
Data Signing — having nodes cryptographically sign the data they provide to smart contracts allows users to identify which nodes sent data and look at their history to determine their performance quality.
Service Agreements — using binding on-chain agreements between the requesting smart contract and the oracle provider that outline the terms of the oracle service and penalties/rewards for performance provides users with enforceable guarantees on the quality of their off-chain data requests.
Reputation Systems — feeding signed on-chain data into reputation systems allows users to make informed decisions about which nodes are good and which nodes are not based on a variety of metrics like successful jobs performed, list of clients served, average response time, etc.
Certification Services — enabling nodes to increase their security and reliability by obtaining any number of certifications can provide users additional guarantees like KYC, geographic location of the node, security reviews of their infrastructure, and more.
Advanced Cryptography and Hardware — providing flexibility for more advanced cryptography (like zero-knowledge proofs) and hardware (such as trusted execution environments) enables oracles to perform additional functions like proving the origin of data (e.g. specific data came from a specific server), keeping data confidential, performing off-chain computation, and more.

In the end

Because of their deterministic characteristics, blockchains require a special method to retrieve data from the outside world. Oracles exist solely to serve as an interface between the blockchain world and the outside world. They allow developers of decentralized services to automate the retrieval of real-world data.

I hope you enjoyed this introductory article, stay tuned for the next edition where we will make our first API calls using oracles. We will explore the most used protocols to date and compare their performance!

That’s it for today, guys! You can always shoot me an email at victorl@sipios.com, connect with me on Linkedin, or access my GitHub projects.

Thank you for reading, happy learning!

Sources

Book: Mastering Ethereum: Building Smart Contracts and DApps

Websites: Chainlink, WINklink, Provable, Ethereum, Blockchain Visual Demo