Go-Ethereum Pending (Wait for It…) Balance

Our vision for Kin is to create a decentralized ecosystem of digital services for daily life.

“For daily life” means that one of our greatest challenges is making blockchain technology accessible to the masses. These days blockchain (mainly crypto-currencies) is known widely, but its use is still reserved to early adopters/tech enthusiasts (more about our challenges bringing blockchain to the masses here).

We stumbled upon one of these challenges recently when we started to develop our Kin mobile wallet SDK. Today, Bitcoin and Ethereum blockchain users, understands that transaction confirmations can take some time due to blockchain consensus protocol, and there are even tools such etherscan.io to monitor transaction statuses. On the other end, mainstream consumers like Kin users will expect transaction to be immediate and will not understand why the balance is not updated instantaneously after spending or earning.

Let’s say Alice started with a balance of 100 kin and just spent 25 kin. We know that if Alice would check her Kin balance immediately after the transaction, she would not see 75 kin. Alice, who probably doesn’t understands how the blockchain works, might get confused. Moreover, Alice might continue to use the app and perform other transactions with Kin, rendering her balance out of sync for a meaningful period of time. This is one of the challenges that needs to be solved by product and engineering to provide the end user with the best experience possible.

We decided to solve this problem by abstracting the transaction details from end users and introducing a pending balance. A pending balance is the sum of the last confirmed balance, plus the sum of pending earned transactions, minus sum of pending spent transactions. Pending transactions are transactions that were broadcasted to the blockchain network that are supposed to be part of the next mined block. In solving the problem this way, Alice can receive immediate feedback without needing to understand the intricate details of the blockchain.

Events/Logs system

The best option with Ethereum standard JSON-RPC is using the “events/logs” system of Ethereum contracts. Events are special feature of the Ethereum-Virtual-Machine (EVM) , The runtime that runs contracts code (Kin itself is a smart token contract). Events are special declared methods that cause called arguments to be stored as logs in a special storage inside of Ethereum blockchain node. From the other side, those logs can be retrieved using JSON-RPC call, through powerful filtering options.

Our targeted event for pending balance, is transfer, which is one of the standard events defined in standard ERC-20 tokens:

event Transfer(address indexed _from, address indexed _to, uint256 _value)

The JSON-RPC API, supports filtering based on the ‘indexed’ params, in our case the from address and the to address. To filter all pending spent transactions from specific user addresses, all you need to do is query based on the from address. Similarly, querying based on the to address will result in pending earned transactions.

The (bumpy) road to implementation

Now that we understands how we should get the pending balance, let’s implement it using go-ethereum in mobile! Well, that’s what I thought…

First, we need to choose the correct method. We have two ways to retrieve logs in Ethereum — registering filter changes (using getFilterChanges) and polling those changes from time to time, or getting logs immediately using getLogs. As we want pending balance, polling changes over time aren’t relevant. We need to get current logs at singular point in time, so getLogs is the way to go. The equivalent method in Geth mobile client is:

EthereumClient.filterLogs(Context, FilterQuery)

FilterQuery param contains all the required params to filter specific log, including setFromBlock() and setToBlock(). Unfortunately, those setters gets BigInt as a param, while JSON-RPC states that it can be either a block number or one of the predefined tags latest/pending/earliest. So how can we specify pending with Geth mobile?

Searching for a clues in the go-ethereum github repo points to an interesting clue. It turns out there are defined consts at rpc/types.go:

These are used in non-mobile code, so I naively try to use this values as a parameter to the aforementioned FilterQuery.setFrom/ToBlock(). This resulted in an exception:

go.Universe$proxyerror: Invalid params: Invalid block number: invalid digit found in string.

This means that Geth treats those constants as negative numbers that lead to illegal block numbers.
Additional research unveils the problem, which is at toBlockNumArg method used when converting those block params to string for Json serialization (eventually Geth using JSON-RPC to send requests to the node):

While Geth does consider the latest special tag with the default (nil) value, there is simply no way to output these predefined tags from input block number!
This made the solution obvious: I need to patch this code and add handling for predefined consts and output them as the matching tags for pending/latest/earliest block. My submitted pull request can be observed here.

So, that’s it, right?

After trying to get filter logs (and to understand the correct params to put for topics params at FilterQuery, but that’s for another time…), I didn’t managed to get any logs at all using getLogs query. To my surprise, using filter changes (getFilterChanges) did work!

After trying different params (and also without using filters), trying different nodes (e.g., infura/self hosted node/main net network/ropsten — test network) and trying different way to query (e.g., posting directly HTTPS request against node), I discovered another issue, this time at the server (node) side. In searching in go-ethereum issues, it’s evidnet that I’m not the only one noticed this issue: https://github.com/ethereum/go-ethereum/issues/15359

So, what do we do now?

Well, Geth is just one implementation of the Ethereum protocol (the official one, but not the single one). Testing again, the getLogs method against Parity node results with expected logs. Therefore, the simple solution for us now is to work against the Parity node instead of Geth node.

Bringing blockchain to mainstream consumers is a challenging mission from every aspect, and our journey (see Oren Zakay’s previous post) with bleeding edge technologies like Ethereum and the go-ethereum library, is filled with obstacles. We enjoy working on these problems and will continue to share with the community.

Note: My journey is specific to Android, but the same fix and conclusions apply to iOS as well. Both implementations can be observed in our open sourced Android and iOS SDKs.