Evolution of consensus mechanisms: from classical methods to local consensus
Part 3: The General Consensus vs. Local Consensus
The consensus mechanism is the cornerstone of distributed technology, such as the blockchain. Since the decentralized nature of systems based on such technologies does not provide for a single center to validate, we must have a mechanism that, on the one hand, allows us to reliably perform this function, and on the other, has a distributed nature. In other words, such a mechanism should provide confirmation of the validity of the transaction each time by reaching an agreement between all network participants (or rather all active participants — full nodes) on this issue, providing protection against a possible double-spend problem, and ideally providing immutability of the blockchain’s distributed ledger.
In the first part of this series of articles, we discussed in detail the classical methods of achieving consensus, in the second part — Nakamoto consensus, used in modern decentralized blockchain systems. Finally, here we look at sharding and local consensus, underpinning the main Layer 2 and Layer 3 multi-hop projects based on state channels and trustlines.
The general consensus vs. local consensus
As we saw earlier, despite a diversity of approaches the ideal method of achieving consensus does not exist (yet?). At the same time, all these methods differ only in what is considered to constitute “complexity” and “work” in each of them. In other elements, they are quite similar: all the consensus mechanisms listed above set the task of reaching agreement on the state of the network, ideally, between all of its participants (or at least of all active ones). The key word here in this is the word “all”.
Precisely herein lies the reason for one of the main challenges of all existing blockchains: namely the scalability and speed of transactions. No matter what consensus algorithm — PoW, DPoS, pBFT or other — a project uses, having a commonly distributed registry, as well as the need to achieve consensus among all active network participants (full nodes), will always impose certain limits on the number of transactions per second.
Is there a way to overcome this basic constraint?
This is quite hard to imagine from the point of view of classic blockchain projects, like Bitcoin or Ethereum. After all, as we said at the very beginning, universal consensus is one of the basic principles on which all distributed blockchain technology rests, and if we move away from it, will we not actually return to centralized systems?
It turns out — no. There are ways to do this by moving away from the principle of universal consensus. One such method is the transition to sharding (network segmentation).
Sharding aims to solve the problems of scalability, delays, and throughput of transactions. This is a concept widely used in databases to increase their effectiveness. A segment (shard) is a horizontal part of a database, each of which is stored in a separate server. This distributes the load and makes the database more efficient. In the case of blockchain systems, with the implementation of the sharding solution, each node will have only a portion of the blockchain data, and not all the information of the entire blockchain. Nodes that support a segment store information only about that segment in a distributed form, so decentralization is still maintained within the segment. However, each node does not store complete information on the whole blockchain, and that contributes to scalability.
This leads to the fact that the Proof of Work algorithm cannot be used in conjunction with segmentation, because how can all participating nodes be involved in checking a transaction if each of them has information only on the segment to which this particular node belongs? Therefore, blockchains that implement sharding use the Proof of Stake consensus algorithm (see Part 2).
However, in general, sharding is a method more suitable for updating directly the blockchain systems of the basic Layer 1 level. But there is a more radical way to solve scalability problems, and also the interoperability issue of different blockchain ecosystems. This means the implementation of off-chain solutions, i.e. a departure from the need to have a single distributed registry, which makes it possible to move to a local consensus instead of a global one.
The essence of a local consensus is that agreement about a particular transaction is not achieved by the entire network, but only between specific participants in this particular transaction, through direct communication between them.
It somewhat resembles a strongly simplified version of pBFT, which we mentioned above, in which consensus is also achieved through communication between the nodes. But unlike pBFT, the local consensus involves essentially only the nodes that participate in a specific transaction. And only the final result of all transactions between these nodes (if necessary) is transmitted for writing to the usual blockchain.
But, of course, we are not talking about the transfer of value only between two neighboring nodes. Indeed, if necessary, you can create chains of such nodes, making multi-hop payments possible.
On such principles and technologies, various multi-hop projects of the Layer 2 and Layer 3 class are built, i.e. special decentralized networks that are overlays over basic blockchain ecosystems (Layer 1), which are designed to solve primarily the above-mentioned scalability problem (Layer 2 — Lightning Network, Raiden, Celer), as well as the interoperability problem (Layer 3 — GEO Protocol, Interledger). Payments in such systems are performed through a chain of state channels and/or trustlines opened between their members.
In order to understand how this actually works, let’s turn to the example of the GEO Protocol. All information about the state of the opened channel between two nodes, as well as the history of all transactions between them, is stored locally on these two nodes and, if necessary, is coordinated between them each time by exchanging the one-time keys from the pre-generated key pool.
Thus, nodes can repeatedly change the balance of their composite channel (in the GEO Protocol, a state channel and trustline combination is used) without needing to refer to other nodes, not to mention all the other nodes of the network. If a problem occurs, there are local conflict resolution mechanisms, but if for some reason they have failed, the nodes can apply for arbitration to special service nodes (the so-called observers).
In the case of channel closure, its final balance is fixed by both nodes and, if necessary (in the case of the use of the state channel), is passed to be written to an external blockchain (or blockchains, if operations with several assets took place).
Conclusion
The nature of the first level blockchain systems (Layer 1) is that in order for them to function properly, they are forced to put up with a multitude of design constraints, many of which cannot be overcome. In a sense, these restrictions are not the only natural but also necessary. The need to reach a general consensus refers precisely to such necessary restrictions.
At the same time, to improve usability, as well as market adoption of blockchain technology, there is a need to create special overlays over basic ecosystems (Layer 2 and 3), whose task is primarily to overcome the problems of scalability and interoperability of various blockchain systems, as well as their integration with existing solutions of the real economy.
The architecture of Layer 2 and 3 systems allows the use of technologies other than those used in conventional blockchain systems. These technologies include the local consensus method.
