Hyperledger, Ethereum and Quorum Nonce Management
What is a nonce?
A nonce is a arbitrary, unique number that is used to prevent replay attacks. The entire transaction, including the nonce, is cryptographically signed with the sender’s private key, thus proving to a miner that the transaction’s sender was in possession of the private key.
Why does a nonce need to be unique?
The uniqueness of a nonce prevents replay attacks. An example of a simple replay attack is if you send 3 ETH to ACME Corp for Rocket Powered Roller Skates. Then the ACME Corp salesman intercepts your transaction via a man-in-the-middle attack and broadcasts the same 3 ETH transaction 5 times in an effort to get 500% more ETH from you. The nonce would prevent this scenario as only 1 of the 5 transactions would be mined.
How does the miner know if the nonce is unique?
This is where the underlying architecture gets really interesting. If the miners needed to keep track of every nonce that every account address had ever used, then that would take up a LOT of storage. On a side note, Hyperledger Fabric and Hyperledger Sawtooth actually do store the random nonces, BUT they mitigate the storage issue by associating the used hashes to the current valid certificates. This way they can purge all of the nonces when a new certificate is issued. See the random nonce in the Hyperledger Sawtooth Python SDK example code below.
Ethereum and Quorum do not need to store the nonces, as they are NOT random. The nonce is the transaction count from the sending address. This completely eliminates the need to store nonces AND removes any possibility of a double spend. These advantages bring a new set of issues as the nonces must be sequential with no skipping of numbers. See the Web3 Python SDK example code below.
Ethereum’s nonce issues
The problem arrises when we try to send several transactions from the same address in a short period of time. In the Web3 Python SDK we call the getTransactionCount method to get the next nonce, but that method is getting the count from our provider, via IPC, Websockets or HTTP, (I am using Infura as the provider via HTTP). The provider count is not updated very fast. Even when we call getTransactionCount with block_identifier=’pending’, we still get the default block_identifier=’latest’ count. The call with the block_identifier set to ‘pending’, still returning the ‘latest’ count is a known issue. The problem is shown below when I try to send multiple transactions using the nonce from the getTransactionCount method. They all use nonce 121, so our script raises an error and only the first transaction gets through.
The nonce transaction manager solution
I rewrote the same python script using Redis to store the current nonce and increment the nonce via my python script, no longer relying on the getTransactionCount method from our provider. You can see it sends out 10 transactions with no issues in less than 5 seconds. Notice the getTransactionCount nonce stays at 151, but we have no problems as our Redis nonce increments correctly.
Managed nonce issues
If we skip a nonce due to an error, all transactions after that will be queued, but not processed (sometimes dropped) until the skipped nonce is used. I increased the nonce count by 1 on the Redis CLI and then ran the same python script for 5 transactions.
You can see below that the first transaction ending in ‘96e09bc’ does not even show up as ‘pending’ on Etherscan.
Now we run nonce 161 through and it gets processed, but the previous 5 transactions have been dropped, so we would rerun the last 5 again. I have also seen the queued transactions process on their own before, right after the skipped nonce is processed.
Duplicate transactions?
I retried nonce 162 to show that you normally get a ‘nonce too low’ error from the SDK via the provider, but even if you didn’t, only 1 of the transactions with that nonce would ever get processed. There is no need to worry about duplicates, if you use the same nonce on multiple transactions.
Conclusion
A nonce management system is a must have, whether you are using Hyperledger, Ethereum or Quorum in an enterprise environment. My simple Redis demo in this post illustrates the basic concept, but does not cover multiple clients using a single address, where you would want to use Redis distributed locking like Redlock.
While I am not consulting on blockchain projects, I have been working on an enterprise transaction management system that will include transaction tracking, analytics, alerts and even automated nonce resetting and retries.
How’d you like this article? If you liked it or learned something, please leave a clap! DarkBlock.io is an enterprise blockchain development company and we’re always taking on new clients. Reach out to me at sheffield@darkblock.io or visit our website at DarkBlock.io!
