Using LiquidOracles on EOS, WAX, Telos — LiquidApps DAPP Network Developer Walkthroughs
LiquidHarmony enables many services with customizable trustlessness. Let’s get some data from a web API into a smart contract, using LiquidX to activate it on an EOSIO sister chain.
What is LiquidHarmony?
Today, we’ll be covering LiquidOracles, which is a function of the LiquidHarmony framework. LiquidHarmony, one of the most powerful features of the DAPP Network, is a customizably trustless way to obtain all kinds of services:
- HTTP(S) Get & Post and HTTP(S)+JSON Get & Post requests
- Nodeos History Get
- IBC Block Fetch (Mainnet, BOS, Telos, Kylin, Worbli, Jungle, Meetone, WAX)
- Oracle IBC (Ethereum, Tron, Cardano, Ripple, Bitcoin, Litecoin, Bitcoin Cash)
- Wolfram Alpha
- Random Numbers
- Stockfish chess AI (demonstrating computation capabilities)
- Other vCPU use cases
- SQL
- Potentially any Dockerized application
The first few uses fall under LiquidOracles, and that’s what we’ll be looking at today. Future tutorials will consider the other options, including developing your own LiquidHarmony plugin.
While we’ll focus on a web oracle today, what we learn here can also apply to vCPU, LiquidRandomness, LiquidSQL, the oracle component of LiquidLink, and more.
LiquidOracles offers numerous unique features, including full control over oracle sources and customizable consensus. Oracles can also update much more frequently than many solutions, while remaining affordable.
LiquidOracles offers numerous unique features, including full control over oracle sources and customizable consensus. LiquidOracles can also update much more frequently, while remaining affordable. Learn how to use LiquidOracles now! (Click to Tweet)
We’ll easily set our own sources of information and our own consensus requirements in this walkthrough. For information on how to use the other LiquidHarmony options, see the Getting Started doc.
Setting up Zeus SDK
You’ll need Node.js version 10 (or NVM with nvm use 10) and the Zeus SDK installed. (Alternatively, you can use Zeus IDE.) If you’re not familiar with how to set up Node+Zeus, check out the first DAPP Network walkthrough.
In short, once Node.js is ready to go, simply run:
$ npm i -g @liquidapps/zeus-cmdAs always, the $ sign just indicates this is meant for the command line. Don’t type it.
Once Zeus is set up, navigate to a convenient directory in which to keep your LiquidOracles walkthrough application. A box included with Zeus SDK will provide us a great starting point for our work today.
Unboxing a Sample App as Our Starting Point
Zeus is a fully-featured blockchain SDK that is similar to Truffle. It’s extensible and box based — which means that the only command you can run anywhere is unbox.
Unboxing a box will then allow you to execute whatever it enables within the resulting directory: things like zeus migrate, zeus compile, zeus deploy, zeus test, zeus create contract, and so on.
Let’s unbox a sample oracle contract so we can get a look at it.
$ zeus unbox oracle-dapp-serviceOnce unboxing is complete, head into the new directory:
$ cd oracle-dapp-serviceThe best place to get a look at how LiquidOracles works is the oracle consumer contract.
$ cd contracts/eos/oracleconsumer
$ nano oracleconsumer.cpp(or, of course, use your preferred text editor or IDE)
Includes and Defines
At the start, we see the usual DAPP Network preprocessor directives that enable our EOSIO contract to use DAPP Network features:
/* INCLUDE ORACLE LOGIC */
#include "../dappservices/oracle.hpp"/* ADD DAPP NETWORK RELATED ORACLE ACTIONS */
#define DAPPSERVICES_ACTIONS() \
XSIGNAL_DAPPSERVICE_ACTION \
ORACLE_DAPPSERVICE_ACTIONS#define DAPPSERVICE_ACTIONS_COMMANDS() \
ORACLE_SVC_COMMANDS()
Other #includes, ACTIONS, and COMMANDS would be placed here if we needed more services, but we’ll only be using LiquidOracles today.
Since the start of our class definition and the dispatcher require some custom code supplied by the CONTRACT_START and CONTRACT_END macros, we need to set a CONTRACT_NAME:
#define CONTRACT_NAME() oracleconsumerWith this, our smart contract is ready to use LiquidHarmony’s LiquidOracles. Documentation for other uses such as Wolfram Alpha or Nodeos History Get is available here.
Customizing the oracleconsumer contract
Let’s build an app that will get the latest total U.S.A. national debt. We might use this to create an interesting tokenomic model, for a prediction market, or for some simpler informational purpose.
To kick our project off, let’s rename our contract:
#define CONTRACT_NAME() debtgetterRename (or copy) the .cpp file and folder from “oracleconsumer” to “debtgetter,” as well.
Customizable Sources
Unlike some other oracle solutions, LiquidOracles allows dApp developers to select whatever oracle sources they would like.
Be careful what APIs you select! By selecting a source, you are trusting that source. The trustlessness of the decentralized network helps you be confident that your oracles are reporting correctly on the source — but your providers have no idea whether the source itself is reliable or not. In some situations, it may be wise to query multiple API sources and compare them.
In addition, websites often go down. Even if you trust your primary oracle source entirely, you’ll want backups. Otherwise, an outage of your source could result in an outage for your app.
While we won’t use multiple sources here, we’ll follow a similar comparison process to ensure our oracle providers are being honest.
OK, back to the code.
A comment immediately following the preprocessor lines explains what the oracleconsumer.cpp contract offers by default:
/*
testget — provide a URI using the DAPP Network Oracle syntax and an expected result, if the result does not match the expected field, the transaction fails testrnd — fetch oracle request based on URI without expected field assertion
*/
Oracle source URLs aren’t actually provided in the default oracleconsumer contract we get when we unbox — instead, test actions are provided, and URLs for our sources can be given via command line (though you’ll need to encode them) or from a JavaScript frontend (using Buffer.from). Running zeus test will run through these sample actions, which you can explore on your own.
While being able to send a URI to a contract action as an argument is useful, it might not reflect what we want to do in a typical application, so let’s place our source URI directly in the contract for this tutorial.
As we mentioned, we’ll be getting the current U.S. government debt. This is available at https://www.treasurydirect.gov/NP_WS/debt/current.
Using the query string ?format=json will save us the minor additional headache of dealing with JSONP. A sample response looks like this:
{"effectiveDate": "March 23, 2020 EDT", "governmentHoldings": 6010453915906.78, "publicDebt": 17507401138252.23, "totalDebt": 23517855054159.01}
Here’s a simple way to get this information into our contract, using LiquidOracles:
CONTRACT_START() ACTION getdebt() {
string url ="https://www.treasurydirect.gov/NP_WS/debt/current?format=json";
vector<char> urlv(url.begin(), url.end());
auto debt = getURI(urlv, [&]( auto& results ) {
eosio::check(results.size() > 0, "no results returned");
auto itr = results.begin();
return itr->result;
});
string debtstr(debt.begin(), debt.end());
eosio::check(false,"The current debt is "+ debtstr);
}CONTRACT_END((getdebt))
The key function here is getURI, which takes two arguments:
- a vector of characters (when using JavaScript, use
Buffer.from("<url>","utf8")) containing the URI to call, and - a callback lambda function that handles the results when obtained.
Here, we return the first result out to our variable debt. Like the URL, each oracle result is a vector of chars, so we need to convert back to a string in order to feed it to eosio::check. Of course, this conversion might not be necessary in a number of contracts, depending on the format and intended use of the data obtained.
Now that we’ve eliminated the testget and testrnd functions, the tests in test/oracle*.spec.js won’t function anymore. They’re easy enough to use as a reference for our own tests. Let’s remove all of the oracle*.spec.js except for oracle-web.spec.js, which we can modify after removing all tests but HTTPS Get:
require("babel-core/register");
require("babel-polyfill");
import 'mocha';const { assert } = require('chai'); // Using Assert style
const { getTestContract } = require('../extensions/tools/eos/utils');
const artifacts = require('../extensions/tools/eos/artifacts');
const deployer = require('../extensions/tools/eos/deployer');
const { genAllocateDAPPTokens } = require('../extensions/tools/eos/dapp-services');var contractCode = 'debtgetter';
var ctrt = artifacts.require(`./${contractCode}/`);describe(`Debt Getter Test`, () => {
var testcontract;
const code = 'test1';
before(done => {
(async() => {
try {
var deployedContract = await deployer.deploy(ctrt, code);
await genAllocateDAPPTokens(deployedContract, "oracle", "pprovider1", "default");
await genAllocateDAPPTokens(deployedContract, "oracle", "pprovider2", "foobar");
testcontract = await getTestContract(code);
done();
}
catch (e) {
done(e);
}
})();
});
var account = code;
it('Debt HTTPS Get', done => {
(async() => {
try {
var res = await testcontract.getdebt({
// action arguments would go here, but getdebt takes none
}, {
authorization: `${code}@active`,
broadcast: true,
sign: true
});
done();
}
catch (e) {
done(e);
}
})();
});
});
Since we renamed our oracleconsumer contract, zeus compile and zeus test will only work if we update contract/eos/CMakeLists.txt, changing:
ExternalProject_Add(
oracleconsumer
SOURCE_DIR oracleconsumer
BINARY_DIR oracleconsumer
. . .to
ExternalProject_Add(
debtgetter
SOURCE_DIR debtgetter
BINARY_DIR debtgetter
. . .Now we’re ready to run zeus test.
Of course our test “failed”, since we threw an exception using eosio::check in order to print out the current debt value. On your own, feel free to modify this test to accept this result as passing.
There’s one problem, though: this full JSON response isn’t what we’re after. We only want the “totalDebt” value.
We could add JSON parsing, yes — but instead, let’s locate what we want using what we know about how the response is structured. We can find where "totalDebt" exists in the response and trim our string down to the value we desire. This will be much lighter on contract resources, anyway:
ACTION getdebt() {
string url ="https://www.treasurydirect.gov/NP_WS/debt/current?format=json";
vector<char> urlv(url.begin(), url.end());
auto debt = getURI(urlv, [&]( auto& results ) {
eosio::check(results.size() > 0, "no results returned");
auto itr = results.begin();
return itr->result;
}); string debtstr(debt.begin(), debt.end());
uint8_t start_loc = debtstr.find("totalDebt")+11;
string debtsubstr = debtstr.substr(start_loc, debtstr.length() — start_loc — 1);
eosio::check(false,"The current debt is " + debtsubstr);
}
Now, zeus test gets us the following:
Once we add multiple oracle sources, we’ll probably need to handle the response of each source differently. Ideally, the URLs and settings for parsing responses could be kept in a settings multi-index table.
The values we receive could also be stored, depending on our use case. For now, we’re not using the debt value for anything yet — in fact, we’re halting execution once we print the value.
Furthermore, this little function has no verification of the results whatsoever — besides making sure there is at least one result. Whatever happens to be the first oracle result is returned, no matter how many oracle providers you are using.
This last issue clearly defeats the purpose of decentralized oracles. Let’s fix it.
Customizable Consensus
Let’s add:
- a minimum number of oracle results
- a requirement that they all match exactly
ACTION getdebt() {
string url ="https://www.treasurydirect.gov/NP_WS/debt/current?format=json";
vector<char> urlv(url.begin(), url.end());
auto debt = getURI(urlv, [&]( auto& results ) {
const MIN_ORACLE_PROVIDERS = 2;
eosio::check(results.size() >= MIN_ORACLE_PROVIDERS, "At least " + MIN_ORACLE_PROVIDERS + " provider results are required for consensus");
auto itr = results.begin();
auto first = itr->result;
++itr;
/* CONSENSUS LOGIC checks all results against first one */
while(itr != results.end()) {
eosio::check(itr->result == first, "consensus failed");
++itr;
}
return first;
});
string debtstr(debt.begin(), debt.end());
uint8_t start_loc = debtstr.find("totalDebt") + 11;
string debtsubstr = debtstr.substr(start_loc,debtstr.length() - start_loc - 1);
eosio::check(false,"The current debt is " + debtsubstr);}
We can change MIN_ORACLES to whatever we’d like to ensure that any action using fewer DAPP Service Providers fails. In this case, we’d like the results from each oracle to match the others exactly, or else consensus (and the action) will fail.
However, the sky is the limit when it comes to customizable consensus. We could be draconian with our oracles, as here. We could ignore dissenters below a certain threshold. Or, we could allow for a tolerance and take an average of values. Any verification standards you can code, you can use.
Note that zeus test provides you with 2 local oracle service providers by default, so setting higher minimums will not succeed unless you customize Zeus’s setup. In the real world, and in customized environments, you can use as many providers as you’d like — and, if you wish, even include your own provider as a watchdog, preventing mass collusion against your app.
Deploying Our Application
In order to deploy this application to any EOSIO test or mainnet, we need to:
- Create or use an account with enough resources on the target network
- Select and stake to DAPP Service Providers
- (if using a chain other than EOS) Link our target EOSIO network account and our staking EOS account
- (optional) Customize our zeus-config.js file
- Compile our contract
- Deploy our contract
We’ll do this for the WAX testnet, while covering information for WAX mainnet, too. Other destinations we could currently select include EOS and Telos mainnet and testnets.
1. Create an account.
A detailed walkthrough of account creation is beyond the scope of this tutorial.
For WAX testnet, go to:
https://faucet.waxsweden.org/create_account?<account>
to make an account and
https://faucet.waxsweden.org/get_token?<account>
to get some tokens. More details are available here, along with support links if you have any trouble.
For WAX mainnet, you can head to https://wax.io/blog/how-to-create-a-wax-blockchain-account for a free account, then send sufficient WAX from another wallet or exchange in order to acquire the necessary resources.
If your contract deployment ends up needing more resources than you have, you will be informed of the requirements when the deployment fails. Read more here.
2. Select and stake to DAPP Service Providers.
To get set up on WAX, we can follow the staking instructions here or walkthrough here, but use EOS mainnet instead of Kylin wherever applicable. This means we’ll need a real EOS account, real EOS tokens, and real DAPP tokens. We’ll stake to a DAPP Service Provider endpoint that is live on WAX: several DSPs are currently available.
It’s easy to stake to DSPs from a DAPP Network Portal.
Important: At this stage in the DAPP Network’s development, you’ll need to have access to an EOS account and some DAPP tokens, as EOSIO networks using the DAPP Network with LiquidX are provisioned on the EOS mainnet. If this isn’t an option for you, you can experiment on the Kylin testnet, where free accounts and faucets for Kylin DAPP are available. The process will be, otherwise, the same as WAX testnet. For detailed instructions on how to proceed, read this walkthrough.
3. (if using an EOSIO chain other than EOS) Link our accounts.
Since we’re using LiquidX to run on WAX mainnet and not running directly on EOS mainnet, we need to take some actions to link our WAX account to our EOS account which is staking to DAPP Service Providers:
a) On EOS mainnet, addaccount:
$ export EOS_ACCOUNT=<stakingEOSaccount>$ cleos -u https://nodes.get-scatter.com:443 push transaction "{\"delay_sec\":0,\"max_cpu_usage_ms\":0,\"actions\":[{\"account\":\"liquidx.dsp\",\"name\":\"addaccount\",\"data\":{\"owner\":\"$EOS_ACCOUNT\",\"chain_account\":\"$WAX_ACCOUNT\",\"chain_name\":\"liquidxxtwax\"},\"authorization\":[{\"actor\":\"$EOS_ACCOUNT\",\"permission\":\"active\"}]}]}"
For WAX mainnet, the chain_name is liquidxxxwax.
b) On the target WAX network, adddsp. The syntax is the same as other actions, but takes two arguments:
- owner {name} — name of consumer contract on new chain. In my case, this is
debtgetter11. - dsp {name} — dsp name on new chain (which may be different from the mainnet name for the DSP). Enter the mainnet DSP’s account into the
accountlinktable on theliquidx.dspcontract to find this name.
c) On the target WAX network, setlink.
cleos -u $WAX_ENDPOINT push transaction "{\"delay_sec\":0,\"max_cpu_usage_ms\":0,\"actions\":[{\"account\":\"dappservicex\",\"name\":\"setlink\",\"data\":{\"owner\":\"$WAX_ACCOUNT\",\"mainnet_owner\":\"$EOS_ACCOUNT\"},\"authorization\":[{\"actor\":\"$WAX_ACCOUNT\",\"permission\":\"active\"}]}]}"See more details on these actions here.
4. (optional) We can update zeus-config.js, which contains endpoint information for a number of EOSIO networks.
For example, one current entry is:
'kylin': {
chainId: '5fff1dae8dc8e2fc4d5b23b2c7665c97f9e9d8edf2b6485a86ba311c25639191',
host: 'api.kylin.eosbeijing.one',
port: 80,
secured: false
}For WAX testnet, we can add in for better compatibility with zeus commands going forward. This is an optional step here, as we’ll deploy our contract directly using cleos.
'wax': {
chainid: 'f16b1833c747c43682f4386fca9cbb327929334a762755ebec17f6f23c9b8a12',
host: 'testnet.waxsweden.org',
port: 443,
secured: true
}Here’s an example of what we’d use for WAX mainnet:
'wax': {
chainid: '1064487b3cd1a897ce03ae5b6a865651747e2e152090f99c1d19d44e01aea5a4',
host: 'chain.wax.io',
port: 443,
secured: true
}5. Let’s make sure we have the latest compiled version of our contract.
$ zeus compile debtgetter
6. Deploy our application!
With your cleos wallet unlocked and your cleos wallet import for your WAX account’s private key complete, run:
$ export WAX_ACCOUNT=<yourWAXaccountnamehere>$ export WAX_PUBLIC_KEY=<yourWAXpublickeyhere>$ export WAX_ENDPOINT=https://testnet.waxsweden.org$ cleos -u $WAX_ENDPOINT set contract $WAX_ACCOUNT ./contracts/eos/debtgetter -p $WAX_ACCOUNT@active
Using some environment variables makes cleos commands much easier to parse and prevents lots of copying and pasting. Here, WAX_ACCOUNT is the account you want to be the contract account. In my case, that’s debtgetter11.
The most common problem you’ll encounter is “insufficient RAM.” For reference, here’s how I bought some more:
cleos -u $WAX_ENDPOINT system buyram $WAX_ACCOUNT $WAX_ACCOUNT "35.00000000 WAX"Once the contract is successfully deployed, we can interact with our app on WAX testnet.
$ cleos -u $WAX_ENDPOINT push action $WAX_ACCOUNT getdebt '[]' -p <account>@activeIn my case, this is as follows. I could have also used WAX_ACCOUNT for this action, as nothing is preventing the contract account from calling its own code.
$ cleos -u $WAX_ENDPOINT push action $WAX_ACCOUNT getdebt '[]' -p bitgenstein1@active(If you get assertion failure with message: required service, you failed to stake to DAPP Service Providers correctly.)
As expected, calling getdebt throws an assertion and prints the current U.S. national debt, obtained via our oracle providers! Now, we could proceed with using this information in our contract, adding a backup API or two for resilience, and further customizing our consensus model.
For more information, see the LiquidHarmony Getting Started page and accompanying docs, or experience with the LiquidPortfolio sample application with zeus unbox portfolio.
LiquidOracles — and LiquidHarmony solutions in general — are a flexible way to get your application the access it needs without compromising on decentralization. In the next walkthrough, we’ll dive into LiquidScheduler, which is a great partner to LiquidOracles for many use cases.
In the meantime, if you have any questions…
