A guide to perform web queries in DAPP (Part 1)
Smart contract itself doesn’t provide web query functions.
“The Ethereum blockchain was designed to be entirely deterministic.” This means, that if I took the whole history of the network, then replayed it on my computer, I should always end up with the correct state.
Since the internet is non-deterministic and changes over time, then every time I replayed all of the transactions on the network, I would receive a different answer.
Determinism is important so that nodes can come to a consensus. If there were a contract that required the number of upvotes on this question, the value could differ from time to time or even place to place, causing nodes in the future or without access to this site to reach different conclusions about the state of the network, thus breaking the consensus.
By requiring that every data input is initiated through an external transaction, we can be sure that the blockchain itself contains all of the information required to verify itself. By using a single contract-level oracle (https://oraclize.it) instead of a network or consensus-level feature, we ensure that there is only one canonical result.”
Instead, we can use oraclize.
It can make HTTP GET and HTTP POST requests, retrieve IPFS data, and leveraging aws instances to do computations on blockchain
Most of the information are included in the doc, but I’ll do a summary here.
How does it work?
https://docs.oraclize.it/#background
One solution is to accept data inputs from more than one untrusted or partially trusted party and then execute the data-dependent action only after a number of them have provided the same answer or an answer within some constraints. This type of system can be considered a decentralized oracle system. Unfortunately, this approach has severe limitations:
- It requires a predefined standard on data format
- It is inherently inefficient: all the parties participating will require a fee and, for every request, it will take time before reaching a sufficient number of answers.
The solution developed by Oraclize is instead to demonstrate that the data fetched from the original data-source is genuine and untampered. This is accomplished by accompanying the returned data together with a document called authenticity proof. The authenticity proofs can build upon different technologies such as auditable virtual machines and Trusted Execution Environments.
A more detailed overview of Oraclize authenticity proofs is provided in the Authenticity Proofs section of this documentation.
This solutions elegantly solves the Oracle Problem:
- Blockchain Application’s developers and the users of such applications don’t have to trust Oraclize; the security model is maintained.
- Data providers don’t have to modify their services in order to be compatible with blockchain protocols. Smart contracts can directly access data from Web sites or APIs.
Oraclize engine can be easily integrated with both private and public instances of different blockchain protocols.
While building the service, the Oraclize team has realized that the concept of authenticity proofs has much broader applicability that initially envisioned. For example, the Oraclize Random Data-source can be used even by traditional gambling applications to ensure users of continuous fairness of operation
General Concepts
Data Source Types
http://docs.oraclize.it/#general-concepts-data-source-types
- URL: enables the access to any webpage or HTTP API endpoint
- WolframAlpha: enables native access to WolframAlpha computational engine
- IPFS: provides access to any content stored on an IPFS file
- random: provides untampered random bytes coming from a secure application running on a Ledger Nano S.
- computation: provides the result of arbitrary computation
Additionaly, there also some meta data source such as:
- nested: enables the combination of different types of data source or multiple requests using the same data source, and it returns an unique result
- identity: it returns the query
- decrypt: it decrypts a string encrypted to the Oraclize private key
URL
The URL data source type enables access to any API or web page on the Internet. It supports both HTTP GET and HTTP POST request.
If only one parameters is specified in the query, the service will default to perform an HTTP GET request.
If a second parameter is specified, then the service will perform an HTTP POST request, posting the second parameter as data.
Note that if the second parameter is valid JSON, then it will be posted as such.
The URL data source type supports TLSNotary Proofs and Android Proofs.
More advanced HTTP capabilities, such as Basic Authentication or OAuth, can be build by leveraging the computation data source type.
IPFS
The IPFS data source type can be used to retrieve the content of a file on the IPFS network.
This datasource expects as sole parameter the IPFS multihash in the query.
For example, the file QmT78zSuBmuS4z925WZfrqQ1qHaJ56DQaTfyMUF7F8ff5o would return `”hello world\n”.
If Oraclize fails to fetch the IPFS content within 20 seconds, the request will fail.
Computation
The computation datasource enables the auditable execution of an application or a script by leveraging a sandboxed Amazon Web Service virtual machine.
The application has to print, on standard output, the result of the computation as the last line before it quits. The result can be up to 2500 characters long.
The execution context has to be described by a Dockerfile, where building and running it should start the main application straight away.
Currently Oraclize only provides one type of auditable instance: a t2.micro instance.
The Dockerfile initialization and application execution should terminate as soon as possible, as the execution time is capped at 5 minutes. (Current Docker version used for computation ds is 1.12.1)
The developer can send to Oraclize the application binary or the script, its dependency and the Dockerfile by creating an archive and uploading it to IPFS.
The query expects as first argument the IPFS multihash of that archive, while the following arguments will be passed to the execution environment as environmental variables, making them accessible by the application.
Parsing Helpers
Oraclize offers XML, JSON, XHTML and a binary parser helpers. Examples:
- JSON Parsing: To extract the last-price field from the Kraken API, the fist parameter of the query json(https://api.kraken.com/0/public/Ticker?pair=ETHUSD).result.XETHZUSD.c.0
- XML Parser: Simply pass XML string from your selected API calls
- HTML Parser: helper is useful for HTML scraping. The desired XPATH can be specified as argument of xpath(..)as shown in the example: html(https://twitter.com/oraclizeit/status/671316655893561344).xpath(//*[contains(@class, ‘tweet-text’)]/text()).
- Binary Helper: It can be useful to extract parts of a binary intermediate result by using the slice(offset,length) operator. The first parameter is the expected to be the offset, while the second one is the length of the returned slice. For example, binary(https://www.sk.ee/crls/esteid/esteid2015.crl).slice(0,300)returns the raw bytes of the first certificate of the linked certificate revocation list.
Connecting to blockchain
Public Chains
- Oraclize is integrated natively with the most widely used public blockchain protocols like the Ethereum, Bitcoin, Rootstock mainnets and testnets.
Private Ethereum-based Chains
- Private, Ethereum-based chains can currently integrate with Oraclize by using the Ethereum-Bridge. Forks of Ethereum such as Monax included. More information on how to use the Ethereum-Bridge will become available in the development tools section or refer to this StackExchange answer describing how to use it alongside testrpc.
Examples
Running dockerfile for ipfs with oraclize:
https://docs.oraclize.it/#data-sources-computation
Querying kraken ticker:
https://github.com/oraclize/ethereum-
examples/blob/master/solidity/KrakenPriceTicker.sol
Encrypting queries
Certain contexts, such as smart contracts on public blockchains, might require a level of privacy to protect data from public scrutiny.
Developers can make encrypted Oraclize queries by encrypting a part (or all) of a query with the Oraclize public key.
The encrypted queries feature may be of interested to developers who want to deploy their blockchain applications of public networks.
For example, if an application leverages data from an authenticated API, it would be dangerous to disclose the API key to anyway who is monitoring the public chain.
Oraclize therefore offers the possibility of encrypting the parameters contained in a query to Oraclize’s public key: 044992e9473b7d90ca54d2886c7addd14a61109af202f1c95e218b0c99eb060c7134c4ae46345d0383ac996185762f04997d6fd6c393c86e4325c469741e64eca9
Only Oraclize will then be able to decrypt the request using its paired private key.
Fee
for the most up-to-date price:
To ease development, Oraclize doesn’t charge a contract for its first request of data done using the default gas parameters. Successive requests will require the contract to pay the Oraclize fee and the ether necessary to pay for the callback transaction.Parsing Helpers
Both are automatically taken from the contract balance. If the contract doesn’t have enough funds in his balance, the request will fail and Oraclize won’t return any data.
A guide to perform web queries in DAPP (Part 2)
A guide to perform web queries in DAPP (Part 3)
