Mainnet Technicals: Parallel pBFT and Layered Lattice Data Structure | Part 2

In part 1, we talked about two of TOP Chain’s major innovations and sources of scaling: Multi-Level Sharding, and integrated Multi-Layer Scaling in layer-0, layer-1, and layer-2. Now, let’s begin to understand TOP’s data structure and consensus mechanism.

Two-Layer Lattice

TOP Chain takes the form of a layered lattice data structure, which is technically a form of Directed Acyclic Graph (DAG). However, unlike other DAG based projects such as IOTA, the lattice data structure is much more organized and suited for a sharded architecture.

There are two different lattice structures being used: the Unit Lattice and the Block Lattice.

Unit-Lattice:

Every transaction record applied to an account is called a unit. Units are linked together to form a chain for each account. This means each account is effectively a mini-chain in itself.

The set of all account chains in a shard forms the unit lattice structure. The benefit is that consensus can be done on the account level simultaneously, which adds another layer of parallelization, and therefore much faster throughput. This is in contrast to using a single chain per shard that validates and records transactions from many accounts one after another.

Block-Lattice:

The Block-Lattice acts as a batching mechanism for consensus and cross-shard synchronization. For each account chain, the last unit specifies all the important information of the account, like the current balance. The most recent units are packaged into blocks, and linked into chains for each account sub-space. These are akin to shard chains, and the set of these chains forms the Block-Lattice.

Completely Parallel pBFT Consensus

You may have heard that TOP uses a PoS-pBFT consensus mechanism. While this is true, it is no ordinary pBFT algorithm. Instead, TOP has built a highly parallelized version of pBFT which works in tandem with the lattice data structure to greatly increase scalability.

In total, there are 4 layers of parallelization.

• Multi-Core and Multi-Thread Parallel Computing:

First is a lock-free parallel framework which partitions multiple threads over multiple cores. With the unit-lattice account chain model, transaction validation can be done in parallel even within a single shard. This means nodes can be participating in multiple consensus groups simultaneously by using multiple cores/threads.

• Multi-Role Node Virtualization

A TOP Chain node can fill multiple roles simultaneously. One physical node can virtualize into multiple roles. For instance, Advanced Nodes in the Routing Network can also act as Validator Nodes in the consensus network. This not only improves scalability, but also security. If a shard suddenly loses Validator Nodes and become vulnerable, Advanced Nodes from the Routing Network can quickly take their spots and ensure there is enough stake to keep the shard from being attacked.

• Sub-Groups Within Shards

Each shard has multiple sub-groups at any given time performing consensus in parallel. Another way of saying this is multi-level sharding.

• Multiple Shards For Parallel Transactions

Finally, there is of course the parallelization from breaking the chain into multiple shards.

So to summarize, the chain is broken into shards, and each shard is divided into sub-groups. Physical nodes within these sub-groups can be virtualized to multiple types of nodes, and use multiple cores/threads to process transactions in parallel. The pBFT algorithm TOP uses was designed with all of these levels of parallelization in mind.

This high level of parallelization will allow TOP Chain to achieve throughput even beyond a simple sharded system, which is necessary if TOP is going to support applications with volumes comparable to centralized apps.