Dynamic Proof-of-Work & Prioritization
Details of the upcoming Quality of Service feature
Nano has had a functioning main network since 2014, providing millions of transactions across hundreds of thousands of accounts, all without a single transaction fee. With other networks fees are added to every transaction, which helps increase the cost of sending spam on the network. But these fees also come with trade-offs, including the emergent centralization forces that Nano is aiming to avoid in the long term.
Emergent centralization due to economies of scale
A defining property of cryptocurrency which separates it from other monetary systems is the focus on being…
So we must find other ways to better align network resource usage with the purposes that Nano is best designed for. The approach to this problem so far has been requiring a small Proof-of-Work to be performed by a single machine, such that the difficulty to produce the work value is above a static, minimum threshold configured in the node. This work value is attached to every block on the network and if it is below the minimum value, the block will be thrown out as invalid.
Using other metrics besides this minimum work value to prevent transactions can be too restrictive for certain use cases, or more resource intensive than desired. As a result, updates in V19 will keep this current metric and update it to be more dynamic and prioritized.
What work difficulty threshold is used?
With this update there will be two different work difficulty thresholds in the node:
1) Floor difficulty value: the minimum work difficulty required, regardless of network conditions. This is returned as “network_minimum” from the new “active_difficulty” RPC call in V19.
2) Current active difficulty value: calculated by the node as the average of work difficulty values seen in transactions being confirmed on the network. This is a rolling average calculated by over 20 periods between 16–36 seconds each (depending on network conditions) and is returned as “network_current” from the new “active_difficulty” RPC call in V19.
When using the node’s built-in wallet, work is pre-cached for any local accounts with the threshold at the floor difficulty value. As transactions are published from the wallet’s accounts, the following checks occur to determine how to handle the necessary work:
- If pre-cache work exists and is above the floor difficulty value, the transaction will be immediately published.
- If pre-cache work doesn’t exist or is below the floor difficulty value (should only occur when work was generated for incorrect frontier hash), new work will be generated for the transaction with the threshold at the current active difficulty value.
Because management of the public and private keys for accounts is required for rework, any “watch only” accounts set up in the node will not have rework done for them.
What happens if I publish a block with a work difficulty lower than the current active difficulty?
Once a transaction is published, the wallet will check the confirmation status of the block in 5 seconds or less. If the block is confirmed, no action is taken and the block no longer will be tracked. If the block is not yet confirmed, it will compare the work difficulty on the block with the current active difficulty on the network (which is changing slowly with every 16-second interval):
- If the block difficulty is higher, no action is taken and it will be evaluated again in another ~5 seconds.
- If the block difficulty is lower, rework is triggered for the block at the higher current active difficulty. This work will be canceled if the block is subsequently confirmed during work generation. The resulting block will be republished to the network.
NOTE: Any systems which do not use the internal node wallet to manage accounts will need to make updates to track confirmation delays and initiate rework as they deem necessary to have their transactions prioritized during heavy traffic times.
How does prioritization come into play?
During network conditions where the available resources of the node aren’t saturated, all transactions will be processed quickly. As conditions cause a backlog of transactions waiting to be processed on the node, the prioritization will ensure those with higher work values are handled first.
This processing prioritization requires any spammers saturating the network to spend additional and increasing work generation resources to attempt slowing transactions down for others. At the same time, anyone sending out occasional transactions can have the work for their blocks increased to jump up in the queue and avoid extended delays.
As nodes work through the backlog, the current active difficulty value will fall when spam blocks are cleared out until it nears the floor difficulty value again (likely once the backlog no longer occurs).
Bounding the active transactions
With this update, we will also be bounding the active transactions, which means during periods of high volume some of the lower difficulty transactions will be dropped out of the confirmation process if not being confirmed.
There are a few behaviors driving the dropping of transactions:
- We track the percentage of active transactions which haven’t been confirmed over 20 confirmation request rounds of 16–36 seconds each (depending on network conditions).
- Depending on network conditions, once 75% (during low volume), 50% (during higher volume) or 25% (during highest volume) of active transactions are of this long unconfirmed type, the dropping of transactions will occur.
- Transactions being dropped are calculated based on their “adjusted difficulty”. This adjustment takes into account the average difficulty seen across any chain of blocks, to prevent a series of lower difficulty blocks with a higher difficulty block at the end from getting dependently confirmed despite lower work levels.
- When these dropping conditions are met, up to two of these long unconfirmed, lowest adjusted difficulty elections will be dropped off when a new election tries to be added in their place. This repeats until the percentage of active transactions isn’t over the applicable threshold (25%, 50% or 75%).
- Any blocks you publish through the node wallet will not be dropped by your own node, but other nodes may drop them due to this feature. If this occurs and causes delays in confirmation, the node can either be set up to manually request a confirmation on the block again or wait for confirmation height checking of account frontiers (every 15 minutes, checking up to 256 unconfirmed frontiers), which will cause elections to be restarted.
- “Watch only” accounts in the node wallet are not protected from having their transactions dropped out of the active transactions queue.
This bounding algorithm was designed to help prevent low difficulty transactions during high volume from filling up the active transactions container, which could, in turn, increase the amount of rework required by others publishing transactions.
What behaviors are expected on nodes and the network?
As large volumes of blocks are published to the network, the following behaviors are expected:
- As transactions reach volumes higher than the nodes can actively confirm, they will start filling up the backlog waiting to be processed.
- Confirmation times will increase as nodes must work through the backlog to vote and broadcast.
- If the volumes are large enough to slow confirmation times to more than 5 seconds, node wallets that have published their own transactions will begin comparing the difficulty of work on those blocks. If too low, they will redo work at the current active difficulty level and republish.
- As blocks are republished with more difficult work they will get prioritized higher in the queue to get confirmed more quickly. Unless spammers do rework on their blocks, they will not be as high a priority, allowing other blocks ahead of them. This will increase the confirmation times for the spam blocks but keep other transactions moving quickly towards confirmation.
- If too many blocks linger in the active confirmation queue for too long, some of the lowest difficulty blocks will be dropped out to make room for others.
- Single-account spam event note: as the more powerful nodes on the network confirm up the spam chain more quickly than others, they will be creating conditions where large sections of the chain could be confirmed together by the less powerful nodes that are working to catch up at a later time. This is because of confirmation of dependent elections where a higher block confirmed will result in all valid successor blocks to also be confirmed. This was added along with Confirmation Height in V19.
Looking up to Confirmation Height
Exploring how high this new feature in V19 Solidus will take us
A Quality of Service feature
The updates described above act as a Quality of Service (QoS) mechanism, reducing the confirmation delays to those infrequently transacting on the network, or those with sufficient hardware to do more frequent transactions. The cost of interrupting network operations with spam transactions escalates with V19 and we will continue making updates like this that help better align network resource usage with what Nano is best designed for.