The Road To SAFE-Fleming (Part 2): Dynamic Membership
As we move towards the release of SAFE-Fleming (Alpha 3), we’re starting to release the jigsaw pieces to create a new decentralised peer-to-peer routing network — the brain of the new decentralised internet. And with people around the world forming thousands of peer-to-peer connections in the recent Crust test, it’s now time to move to the next stage.
What’s Dynamic Membership? And What’s it Got To Do with PARSEC?
When we released the White Paper for PARSEC, our ground-breaking ABFT consensus algorithm back in May 2018, we were driven by certain design principles. First, any consensus mechanism had to be scalable in order to support a vast amount of data travelling across a global network. It is also mathematically proven to always reach consensus even under the worst theoretical byzantine threats that any ABFT algorithm can sustain.
Since then, we’ve been building PARSEC into the brain of the Network. Known as the Routing layer, it’s where the decisions get made. And in a big part of this work was devoted to tackling one question in particular: getting PARSEC working in a dynamic, as opposed to a static, Network.
Static v Dynamic. Permissioned v Permissionless
Let’s unpack that for a moment. What’s the difference between a static and dynamic network?
A static network just means that the Network is made up of nodes that don’t change. Perhaps they’re members of the same organisation. Or different companies who are joining together to share the benefits in some way. Regardless of the reason, a decision has usually been made to restrict membership and the system is designed and optimised only for those nodes that have made the cut.
This is very different to the dynamic membership — something that you get with the SAFE Network where one of the guiding fundamentals is to be open and permissionless. Anyone around the world must be able to contribute their computing resources and join the Network. In other words, nodes can join and leave at will.
No gatekeepers.
But with a decentralised network, that presents a number of challenges. What’s stopping an attacker spinning up thousands of nodes to attack the Network (known as a Sybil attack)? What if low-quality nodes that don’t meet the basic requirements join and damage the Network functionality? These, together with other concepts (such as Disjoint Sections, encryption, churn and Secure Message Relay) have all been discussed before and we’ll also be highlighting these in more depth in future posts.
But for the purpose of this article, we’re just going to focus on one: Dynamic Membership. Because it’s a crucial question: with the doors open, how do nodes reach agreement on what happened, and in what order, whilst nodes are joining and leaving at will? And how can a new node possibly find out what’s been agreed in the past in a secure and verifiably correct way?
Listen Carefully — To Those Who Can Prove They’re Telling The Truth
So each node is listening intently to see what’s happening across the Network. But because of the vast size of the Network, messages inevitably arrive at different nodes around the world in different orders. And that’s what you need PARSEC for — to guarantee that every message shared on the Network is recorded — and in the same order — by every other node.
Check out the following video to learn more about how PARSEC works with Dynamic Membership — in other words, nodes joining and leaving the Network at will. It shows an example which you can run for yourself (head along to https://github.com/maidsafe/parsec) in order to see graphs that illustrate the process for you. The example shows exactly what happens as each round of gossip is generated and how different nodes then take this information to arrive at the same sequence of stable blocks — despite the fact that nodes are constantly being added and removed.
TL;DR it doesn’t matter if the membership of the Network is constantly changing. The SAFE Network will still reach consensus.
So what happens when a node has poor connectivity or the other nodes realise that it’s malicious and attempting to attack the Network? The nodes all vote using PARSEC. And if consensus is reached, they remove the Node from their list of peers, forcing the node to leave. It’s a pariah. No honest node will accept any of its messages. It has no option but to disconnect.
PARSEC and Joining the Network
PARSEC is also used by any node that wants to join the Network. A node following PARSEC notifies others that there’s a new node who wants to join. Voting time once again — and, when the existing nodes reach consensus, the new node joins.
But how does the new node catch up with what’s been happening? Where does it go to learn the existing history of the Network?
This is the really interesting bit. After being voted in, the new nodes get sent the whole gossip graph of the Network at that point in time. So the newcomer can now trace the history of consensus on the Network all the way from its genesis to the present day. It can use this to derive a provably correct list of current members by replaying every single membership change that’s taken place since the start of that Section of the Network.
And now the graph’s updated — to show that the new Node has joined the Network and is a full member.
Conclusion
If you’re building permissionless networks, this is a well-known challenge. When you can’t use something like a proof-of-work blockchain (because the restrictions of throughput and encryption restrictions in blockchain tech don’t scale for a project like ours), the answer isn’t straightforward. But now hopefully you can see how the SAFE Network provides a solution.
Of course, this is a part of the puzzle — not the final solution. But with each passing day, we’re getting closer to that all-important SAFE-Fleming release. Next up, we’ll be diving into Malice Detection and Secure Message Relay, amongst many other concepts — but we’ll leave that for another day…
If you’ve got any questions, just give us a shout — either in the comments below, on the Forum or on any of the usual social media channels.
As ever, thanks for reading!
