On Saturday, the Nym testnet underwent what’s known as a Sybil attack, where someone multiplies resource use in order to annoy a network into submission. From 11am until evening, an internet comedian was auto-scripting new nodes, so we went from about 1,600 nodes to well over 10,000 in the space of a few hours (bigger than Tor, and as big as the larger cryptocurrency systems in the world). At that point, we shut down the network monitor, so that in case the attack continued we didn’t have monitor timeouts and start accidentally penalizing good nodes.
Rewards for node operators will still be given out as normal, but the network monitor which increases reputation is currently disabled. We have also temporarily shut off registration of new nodes for the moment, to get rid of the Sybil attack.
This was a good stress-test of the network, we scaled very well throughout it.
What’s the plan now?
We are going to put in place a temporary fix which will allow us to re-enable the network monitor. We are also going to do a few other things:
a) we are going to cap the network for the moment at 1500 mixnodes. This is slightly less than our node count on Saturday when the attack happened, and it seems like a good network size for now. As a side-effect, our comedian friend will see no benefit from the attack. If a node under-performs (drops packets, or is off line and its reputation drops) the node will be removed from the network and a new node can join.
b) we are going to enforce a one-port-per-IP-address rule on all nodes. There is no extra CPU power or bandwidth available to the network if people set up multiple mixnode instances on one machine and run on multiple ports. People who do it shouldn’t get any extra rewards.
c) we are also going to stop accepting new validators.
So, most parts of the network will be capped in size. We would, however, like to see some more gateways join the network. Gateways will in the near future be included in the network monitor’s assessment, and will gain rewards in line with reputation points just like mixnodes currently do.
Once those changes are made, we will restart the network monitor.
How will mainnet deal with these kinds of attacks?
Sybil attacks are a well-known and understood problem in decentralized systems design. As we progress towards mainnet, the plan is that mixnodes will need to “stake in” with some amount of token. If a node subsequently performs poorly, then it will either not get rewards (leading to an opportunity cost), or it might have its stake slashed. We are still debating the exact mechanics here.
We are also planning a system of stake delegation for mixnodes. Taken all together, the system will aim to consolidate a group of high-quality nodes that are able to handle network traffic, maintain a good margin of spare capacity, and provide staking rewards for both mixnode operators and users who delegate.
We are still building the necessary staking contracts, so it’ll take a few weeks until there is a permanent solution to the problem which allows the network to keep growing.
As a side note, to whoever set up the scripting attack: thanks! You did a nice little scalability test for us. The attack proved that we were able to scale several of our systems (the network monitor, our trial mixnodes, the directory embedded within our validator, and the network explorer) well beyond what I would have been confident about beforehand.
We probably need a redesign of the network explorer so that we’re not showing all nodes, something that has been apparent for a long time. But the basic functioning of many core systems was mostly unimpaired due to the new architectural changes in release 0.9.x. Expect further network survivability improvements in 0.10.x and beyond.