Hyperledger, Ethereum and Quorum Nonce Management

Sheffield Nolan
Oct 15, 2018 · 5 min read
Image for post
Image for post
“Matrix movie still” by Markus Spiske on Unsplash

What is a nonce?

A nonce is a arbitrary, unique number that is used to prevent replay attacks. The entire transaction, including the nonce, is cryptographically signed with the sender’s private key, thus proving to a miner that the transaction’s sender was in possession of the private key.

Why does a nonce need to be unique?

The uniqueness of a nonce prevents replay attacks. An example of a simple replay attack is if you send 3 ETH to ACME Corp for Rocket Powered Roller Skates. Then the ACME Corp salesman intercepts your transaction via a man-in-the-middle attack and broadcasts the same 3 ETH transaction 5 times in an effort to get 500% more ETH from you. The nonce would prevent this scenario as only 1 of the 5 transactions would be mined.

Image for post
Image for post

How does the miner know if the nonce is unique?

This is where the underlying architecture gets really interesting. If the miners needed to keep track of every nonce that every account address had ever used, then that would take up a LOT of storage. On a side note, Hyperledger Fabric and Hyperledger Sawtooth actually do store the random nonces, BUT they mitigate the storage issue by associating the used hashes to the current valid certificates. This way they can purge all of the nonces when a new certificate is issued. See the random nonce in the Hyperledger Sawtooth Python SDK example code below.

Ethereum and Quorum do not need to store the nonces, as they are NOT random. The nonce is the transaction count from the sending address. This completely eliminates the need to store nonces AND removes any possibility of a double spend. These advantages bring a new set of issues as the nonces must be sequential with no skipping of numbers. See the Web3 Python SDK example code below.

Image for post
Image for post

Ethereum’s nonce issues

The problem arrises when we try to send several transactions from the same address in a short period of time. In the Web3 Python SDK we call the getTransactionCount method to get the next nonce, but that method is getting the count from our provider, via IPC, Websockets or HTTP, (I am using Infura as the provider via HTTP). The provider count is not updated very fast. Even when we call getTransactionCount with block_identifier=’pending’, we still get the default block_identifier=’latest’ count. The call with the block_identifier set to ‘pending’, still returning the ‘latest’ count is a known issue. The problem is shown below when I try to send multiple transactions using the nonce from the getTransactionCount method. They all use nonce 121, so our script raises an error and only the first transaction gets through.

Image for post
Image for post

The nonce transaction manager solution

I rewrote the same python script using Redis to store the current nonce and increment the nonce via my python script, no longer relying on the getTransactionCount method from our provider. You can see it sends out 10 transactions with no issues in less than 5 seconds. Notice the getTransactionCount nonce stays at 151, but we have no problems as our Redis nonce increments correctly.

Image for post
Image for post

Managed nonce issues

If we skip a nonce due to an error, all transactions after that will be queued, but not processed (sometimes dropped) until the skipped nonce is used. I increased the nonce count by 1 on the Redis CLI and then ran the same python script for 5 transactions.

Image for post
Image for post

You can see below that the first transaction ending in ‘96e09bc’ does not even show up as ‘pending’ on Etherscan.

Image for post
Image for post

Now we run nonce 161 through and it gets processed, but the previous 5 transactions have been dropped, so we would rerun the last 5 again. I have also seen the queued transactions process on their own before, right after the skipped nonce is processed.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Duplicate transactions?

I retried nonce 162 to show that you normally get a ‘nonce too low’ error from the SDK via the provider, but even if you didn’t, only 1 of the transactions with that nonce would ever get processed. There is no need to worry about duplicates, if you use the same nonce on multiple transactions.

Image for post
Image for post

Conclusion

A nonce management system is a must have, whether you are using Hyperledger, Ethereum or Quorum in an enterprise environment. My simple Redis demo in this post illustrates the basic concept, but does not cover multiple clients using a single address, where you would want to use Redis distributed locking like Redlock.

While I am not consulting on blockchain projects, I have been working on an enterprise transaction management system that will include transaction tracking, analytics, alerts and even automated nonce resetting and retries.

How’d you like this article? If you liked it or learned something, please leave a clap! DarkBlock.io is an enterprise blockchain development company and we’re always taking on new clients. Reach out to me at sheffield@darkblock.io or visit our website at DarkBlock.io!

Get Best Software Deals Directly In Your Inbox

Image for post
Image for post

Coinmonks

Coinmonks is a non-profit Crypto educational publication.

Sign up for Coinmonks

By Coinmonks

A newsletter that brings you week's best crypto and blockchain stories and trending news directly in your inbox, by CoinCodeCap.com Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Sheffield Nolan

Written by

Blockchain Architect Consultant at https://darkblock.io/

Coinmonks

Coinmonks

Coinmonks is a non-profit Crypto educational publication. Follow us on Twitter @coinmonks Our other project — https://coincodecap.com

Sheffield Nolan

Written by

Blockchain Architect Consultant at https://darkblock.io/

Coinmonks

Coinmonks

Coinmonks is a non-profit Crypto educational publication. Follow us on Twitter @coinmonks Our other project — https://coincodecap.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app