Idena hard fork announcement: Validation sharding

Dear Idena community,

We plan to release a new version of the Idena node on September 16, 2021. It will be a seamless hard fork update.

There are the following changes planned.

Sharding Phase I (Validation sharding)

Sharding Phase I, or Validation sharding, will include the following changes in the protocol:

1. Network segmentation into shards

• The network is dynamically split into N shards, where N = 2ᵏ depends on the network size (all identities including Suspended and Zombies are counted). The size of the shard is set from 2400 to 5000 identities. Shards quantity and size are adjusted after each validation ceremony according to the transition table:

Let’s consider an example. If according to the validation results, the network size is in the range of 10000..20000 identities, then 4 shards will be allocated. If, after the validation, the network size has grown to more than 20000 identities, then 8 shards will be allocated. Then, if the size of the network has reduced to less than 19200 identities, then the number of shards will be reduced to 4.

• Each validated address is assigned with a shard number;
• Validated nodes prioritize connections to the nodes in their own shards. At the same time, connections to random nodes from the other shards are also possible.
• Unvalidated and shared nodes operate in multishard mode: they connect to any other nodes ignoring their shard number.
• Identities with the Candidate status are assigned with a shard number when they activate an invitation.

2. Validation sharding

The validation is held within every shard independently of the other shards. The flips are also distributed within the shard. The submit answers transactions are broadcasted only inside the shard as well.

Successful validation criteria:

• More than 50% of the shard receives short session answers from the user before 13:32:00 UTC
  For flip authors: more than 50% of the shard receives the keys for decoding the flips by 13:30:30 UTC

Traffic isolation during the validation ceremony: the traffic inside the shards is isolated in such a way that allows reducing the load on each channel of every node.

• Messages related to your own shard are broadcast only to peers of your own shard
• Incoming messages from peers from other shards are relayed with low probability.

Validation completion criteria:

• 5 blocks without validation transactions related to each of the shards are mined in a row

Motivation

1. Validation scaling. During the validation, all nodes receive messages from all users. This will inevitably lead to problems for large network sizes. Sharding allows to split the network into segments and isolate traffic within each segment. Any segment of the network has no information about the validation in any other segment of the network.
2. Reduction of the size of validation transactions (EvidenceMap).
3. Sharding Phase II preparation. Dividing the network into segments is a preparatory step for dividing the address space, where the state of each wallet is determined not by the entire network, but by a separate shard.

Sharding migration plan

1. The fork will be activated no earlier than September 23rd. Thus, the validation on September 21 will be held as usual (no shards).
2. The fork will be activated in epoch #0075. After successful activation of the fork, the network will start working in a single shard mode.
3. The network will split into shards only in epoch #0076.
4. The first validation in sharding mode will be between epoch #0076 and epoch #0077.

More invitations for the foundation address

The number of invitation codes that can be generated by the foundation address (0xcbb98843270812eece07bfb82d26b4881a33aa91) is increased from 500 to 10% of the network size after validation.

Motivation

The number of god node invitations is increased to offer more invites to Idena ambassadors, influencers, and community leaders who promote Idena. These affiliates can take part in the Idena Affiliate Program and earn rewards for the successful validation of their candidates.

Flips keywords dictionary change

The dictionary is reduced from 3300 words to 640:

• most of the non-visual words are excluded
• most of the synonyms are excluded
• words that provoke the creation of typical flips (words related to hospital, education, prison, daily routine …) are excluded
• Idena-related words are added
• word combinations with emotions are added to the category of people

The new dictionary will be valid from epoch #0076.

Acknowledgments: Mahmoud for helping to clean up the dictionary.

We are open to suggestions on adding new keywords to the dictionary, which will contribute to a greater variety of created flips.

Motivation

• Improve the quality of the created flips.
• Improve reporting criteria.

Increasing reporting rewards

We have established a separate reward fund for successful reporting by reducing other reward funds.

Applying to the latest validation, the reward for one successful report would have been 38.66 iDNA.

Motivation

The quality of flips depends on how the network can eliminate bad flips (for example, flips that do not match the keywords). Reporting is an important part of the protocol and requires the involvement of the super-majority of the network.

We can not decrease the reporting threshold to less than 50% because it could lead to false reporting. Therefore, we suggest financial motivation to increase user reporting activity.

Increasing rewards can lead to over-reporting of flips. Therefore, the number of reports per user is limited. As the number of reports grows, the rewards for successful reporting reduce. We expect that when the users are more active and more responsible with reporting, it will lead to better flip quality.

Smaller flip reporting committees

When an insufficient number of users is included in the flip committee, the requirements for flip reporting are toughened:

• With a committee of 3 users, 100% of reports are required to report a flip
• With a committee of 4 users, 3 out of 4 reports are required to report a flip
• With a committee of 5 users, 4 out of 5 reports are required to report a flip
• With a committee of 6 or more users, 50% +1 reports are required to report a flip

Motivation

Eliminate false reports in small committees

Other changes

• Remove transitive delegation for suspended identities: mining delegation will be removed for any identity that has regained their active status from suspended or zombie;
• Prevent KillTx transactions for users with Undefined status.
• Shared node profile is added to support sharding. If you run a shared node please use the following parameter:

idena-node-linux-0.27.0 --config=config.json --profile=shared

The parameter can be used no matter if the hard fork is activated or not.

How the hard fork will be activated

1. The new node version will support both the current blockchain protocol and the upcoming protocol changes.

2. Running the new version of the node means that the validated address is voting for the upcoming protocol changes. A node with an active mining status will be automatically publishing signals about the hard fork readiness.

3. The hard fork update will be activated only when the following voting criteria are met:

• more than 80% of online mining nodes support the upcoming changes
• more than 66.6% of validated addresses support the upcoming changes

Delegated addresses are not counted. Pool owners are counted once.

4. The hard fork update will be activated no earlier than September 23 or anytime after once the voting criteria are met. Otherwise, the hard fork update will be blocked on September 30.