Reducing ETH Gas by making an Asynchronous Tx with Oraclize
In a previous article (š¤) I was able to reduce an Ethereum transaction costing 95 Million (MM) gas down to 4.1MM by converting arrays to byte strings. This was a big step in the process of building clovers.network but 4.1MM gas was still unacceptable. I was able to reduce it again to 1.5MM by utilizing an Oracle to offload the bulk of the work and save only the resultābasically making an asynchronous call on the Ethereum Virtual Machine (EVM).
The transaction in question contains a function that plays a game of Reversi using moves supplied by the user. If the game is valid and hasnāt previously been registered, the user becomes the owner of that board and is able to sell it as a Clover (ā¤). Furthermore, if the board is symmetrical then the user receives a mining reward in ERC20 ClubToken (ļøā£ļø) relative to the rarity of the symmetry. While the game is rather simple to program on the EVM the level of complexity is still very expensive. Thatās because every step in the process of checking the game is saved along with the result of that game. This is important to prove the method of validation, however thereās another way to prove validation while not having to pay for it: ask an oracle š®.
An oracle provides a portal to the world outside of the EVM. If you want to know the current price of Ether in USD, Euro or GBPāask an oracle. If you want to know the weather š¤ in Chicago , who won the Cubs š» game or whether your flight āļø to ORD is delayedāask an oracle. You can also do things with an oracle that arenāt possible on the EVM like generate random numbers.
Thereās some debate about whether these features belong on the Ethereum Blockchain, since in theory all transactions should be verifiable and repeatableāhow can a URL request at a specific moment in time be repeatable? (For more information about that debate and oracles in general look here, here and here.) Luckily for me I wanted an oracle to call a function already on the EVM. That way the method of validation is still verifiable but I donāt have to spend gas recording all the steps producing the result.
He Said She Saidā¦
Oraclize is an oracle that provides a great selection of datasources including Wolfram Alpha, IPFS and any publicly accessible URL. They also offer the ability to query various blockchains for basic info like block number and mining difficulty. I needed to run a non-transactional (constant) function on the Ethereum Blockchain using eth_call
but Oraclize doesnāt offer it at this point. Instead Iāll use Infuraās publicly available Ethereum node to make a JSON RPC transaction (same way as Metamask does it). Before I get to building the JSON RPC POST request letās look at the contract so far:
function claimGame(bytes28 firstMoves, bytes28 lastMoves) {
if (isReal(firstMoves, lastMoves)) {
saveGame(firstMoves, lastMoves);
}
}function isReal(bytes28 p1, bytes28 p2) constant returns(bool) {
// play the game and check for completeness and errors
...
}function saveGame(bytes28 p1, bytes28 p2) {
// finally save the game
...
}
Here checkGame()
is a function with game moves as parametersāin this case the moves are stored in bytes28 format (for a similar technique storing data in bytes check out the arrays to bytes article I mentioned earlier). The first thing checkGame()
does is check if the moves play a real game by using isReal()
. This is the expensive part of the contract that Iām trying to avoid. However youāll notice that it is a constant function, meaning it doesnāt change anything on the blockchain. Thatās why it would be possible to call it with an oracle, who wouldnāt need to pay any gas to do so. Afterwards the oracle can send the results back to the contract to be saved cheaply.
Adding Oraclize
The first step to utilizing Oraclize is to add their contract to yours. You can download a copy of oraclizeAPI.sol from their github. Add it to the top of your contract and let your contract inherit the functions. In this case Iām calling my contract CheapTrick.
pragma solidity ^0.4.13;import "./oraclizeAPI.sol";
contract CheapTrick is usingOraclize {
...
}
The next step is triggering the Oracle with the designated URL datasource. The contract has inherited the function oraclize_query
which takes a variety of different parameters depending on your needs. Weāll be using the format that takes the first param as the data source, the second param as the URL endpoint and the third parameter as the POST object to be sent along with the request. Alternatively you could add an integer representing the number of seconds to wait before triggering the request, and an explicit amount of gas to be used in the callback.
function claimGame(bytes28 firstMoves, bytes28 lastMoves) {
oraclize_query('URL', 'https://infura.io', '{...}');
}
The final step is calling the inherited __callback()
function to handle the results of the URL datasource query.
function __callback(bytes32 queryId, string results) {
if (results == 'true') saveGame(???, ???);
}
Youāll notice in this callback youāve lost the reference to which moves were being played. Oraclize provides a query ID to help with that process. In order to keep track of which callback belongs to which query you can keep track of them with a mapping and a struct like this:
pragma solidity ^0.4.13;import "./oraclizeAPI.sol";
contract CheapTrick is usingOraclize { struct Moves {
bytes28 firstMoves;
bytes28 lastMoves;
} mapping (bytes32 => Moves) validIds;
// oraclize_query returns the query ID that is used in the mapping
function claimGame(bytes28 firstMoves, bytes28 lastMoves) {
bytes32 q = oraclize_query('URL', 'https://infura.io', '{...}');
validIds[q].firstMoves = firstMoves;
validIds[q].lastMoves = lastMoves;
} function __callback(bytes32 q, string result) {
if (bytes(result)[65] == 0x31) {
saveGame(validIds[q].firstMoves, validIds[q].lastMoves);
}
} ...}
In this scenario the query IDs are saved in a mapping of a struct using the query ID as a key. When the callback is triggered the moves can be extracted again using that same query ID.
String Theories
You may have also noticed or been confused by the line if(bytes(result)[65] == 0x31)
. This is the real way to perform if (results == ātrueā)
which was used falsely earlier. The oraclize_query()
hits a JSON RPC endpoint which should in turn call the previously seen isReal()
function. This function returns a boolean but since Ethereum works in increments of bytes32 that bool value is returned as bytes32. Instead of returning the string ātrueā it returns the hexadecimal value of 1.
true = 0x0000000000000000000000000000000000000000000000000000000000000001false = 0x0000000000000000000000000000000000000000000000000000000000000000
This is further complicated by the fact that the result in__callback()
is actually a string. So itās not returning bytes32
but rather bytes32
as represented by a string
.
true = "0x0000000000000000000000000000000000000000000000000000000000000001"
false = "0x0000000000000000000000000000000000000000000000000000000000000000"
In order to detect whether the game is valid or not we need to look at the last value in that string and detect if it is a 1
or a 0
. While working with strings in Solidity itās important to remember that they are stored as byte arrays (bytes[]
) of UTF8 characters. According to w3schools.com the UTF8 control characters in our string look like this:
string 0 = decimal 48 = hex 0x30
string 1 = decimal 49 = hex 0x31
string x = decimal 120 = hex 0x78
In Solidity our string as represented in bytes[]
would look something like this:
string = "0x0000000000000000000000000000000000000000000000000000000000000001"string[] = ["0", "x", "0", "0", ..., "1"];
bytes[] = [0x30, 0x78, 0x30, 0x30, ..., 0x31];
We need to check the last element in the array so we use the same snippet from the contract above: bytes(result)[65] == 0x31
(remember the array has a length of 66
due to the 0x
preface) and voila we detect whether the result was true or false.
At this point it may be good to point out that Oraclize also offers the ability to upload a snippet of custom code to IPFS with a docker configuration that would allow it to be deployed on an Amazon micro server long enough to be run with the result returned instead of a URL datasource. If this were done the return string could be more efficient than the one we get from the RPC endpoint. However a URL datasource costs the contract owner ~$0.01 per query and the micro server costs ~$0.50 per request. (If youāre still reading this youāll know by now that Iām always lookin for those deals š¤)
JSON RPC POST
For the last part itās important to see what was actually inside of that JSON RPC POST object sent to the Infura endpoint which was earlier represented by the nefarious {ā¦}
. This means that we need to craft our transaction manually based on the JSON-RPC specs here and crafting the data object following the Ethereum Contract ABI specs here. Our basic eth_call
function follows this format:
// Request
curl -X POST --data '{"jsonrpc":"2.0","method":"eth_call","params":[{coming soon}],"id":1}'
// Result
{
"id":1,
"jsonrpc": "2.0",
"result": "0x"
}
The params array consists of an object with the contract address and the data being sent, plus the desired block number:
{"jsonrpc": "2.0", "method": "eth_call", "params": [{to:"0xFAK3W4LL374DDR355", data:"...."}, "latest"]}
The data value will be a hexadecimal representation of the desired function name and the parameters being sent along with it. As per the specs, the function name is represented by the first 4 bytes of the hash of the string of the name of the function (including any parameters š³). In our case using the web3.js utils to help, it would look as follows:
var utils = require('web3-utils')let functionName = "isReal(bytes28,bytes28)"functionName = utils.sha3(functionName)
//0x6b3bd7986bb57b171ccf6056a91eae803767c4600238e08445ece9b98c39ca21functionName = utils.hexToBytes(functionName)
// [107, 59, 215, 152, 107, 181, 123, 23, 28, 207, 96, 86, 169, 30, 174, 128, 55, 103, 196, 96, 2, 56, 224, 132, 69, 236, 233, 185, 140, 57, 202, 33]functionName = functionName.slice(0, 4)
// [107, 59, 215, 152]functionName = utils.bytesToHex(functionName)
// 0x6b3bd798
As a result our function name looks like 0x6b3bd798
. The next part is making byte representations of the parameters. Since our moves are already in hexadecimal format we just need to adjust them from 28 bytes to 32 bytes by padding them and removing the 0x prefix:
var utils = require('web3-utils')let firstMoves = "0xd9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee5"first32Moves = utils.padRight(first32Moves.slice(2), (32 * 2))
// d9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee500000000let lastMoves = "0xbd9bb7ed12e559bfcaad69b5f04fa1061438927fc681167470000000"lastMoves = utils.padRight(lastMoves.slice(2), (32 * 2))
// bd9bb7ed12e559bfcaad69b5f04fa1061438927fc68116747000000000000000
Put them all together and weāve got our data š
{"jsonrpc": "2.0", "method": "eth_call", "params": [{to:"0xFAK3C0N7R4C7W411374DDR355", data:"6b3bd798d9b7774f9af573c5d69d4996a971f147dfac39f7e9f37785891dfee500000000bd9bb7ed12e559bfcaad69b5f04fa1061438927fc68116747000000000000000"}, "latest"]}
You can test the results using Oraclizeās great query tester here .This example code doesnāt correspond to a deployed contract so will not actually work. However, if you follow the link youāll see a working example requesting the current block number. VERY IMPORTANT: Donāt forget to add a space character at the beginning or end of the POST payload. This tells Oraclize that the data is in fact a POST object. I had a lot of trouble with that until someone from the Oraclize team answered my github issue š
In Conclusion
In total weāve covered:
- What is an oracle
- How to import and use Oraclizeās API contract
- How to keep track of the query and the callback
- How to work with strings in Solidity
- How to craft a JSON RPC eth_call to the Infura endpoint