Scaling and Sharding at Insolar

Insolar Team
Feb 6, 2019 · 5 min read

A combination of node role allocation and sharding allows the Insolar blockchain platform to reach transaction speeds in excess of 10,000 transactions per second.

Transaction speeds in the blockchain arena are a topic of great debate, with platforms thus far unable to scale so that a large number of transactions can be executed across a large decentralized network. The problem is that previous decentralized systems become centralized when scaling up in order to maintain consensus across the global network state. As such, blockchain networks thus far have therefore been limited by the DCS Theorem, which says that only two properties can be simultaneously achieved out of Decentralization, Consensus and Scale. The theorem is true but it can be “cheated” with proper structuring of the problem.

Insolar testnet Blockchain Explorer, demonstrating rapid transaction speeds

How does Insolar scale whilst remaining decentralized?

Insolar’s blockchain platform has overcome the above mentioned theorem by dividing the overall problem into 3 sub-problems, where each sub-problem is solved by using different DCS properties and at a different time.

Additionally, Insolar recognizes that there is no “one consensus to rule them all”. The choice of consensus must be based on purpose, and so should be able to vary for different transactions.

To solve this challenge, Insolar has introduced 3 major phases:

  1. Network Consensus based on the Globula Network Protocol

a. It covers the decentralization and consensus properties of the DCS Theorem and ensures that whole network agrees on:

  • who are active network members; and
  • on entropy (randomness).

b. This consensus is pure BFT and its results are valid for the duration of a pulse (block time).

2. Work Partitioning

a. Starts when Network Consensus is reached and covers the consensus and scale properties of the DCS Theorem.

b. By applying a deterministic function over the uniform view on active members list and entropy (produced by Network Consensus), Insolar gets a work distribution map that is identical on every active member.

c. This work distribution map enables Insolar to minimize workload by applying only one executor node per smart contract.

3. Validation and Per-transaction Consensuses

a. Secures the network from malicious nodes when handling contracts and enables per-transaction consensus.

b. Allocation of a node as executor to smart contracts expires at the end of the pulse, and validators are selected to validate results at the next work partitioning phase. This guarantees that executors are unable to predict the validators that will verify their work.

c. In other words, we run thousand of small consensuses for each smart contract updated, and each consensus can be adjusted individually.

So, by considering time and functional phasing, Insolar overcomes DCS limitations and provides unique capabilities. But this is not the only element of the OmniScaling feature of Insolar, and there are more elements that bring about increased security and performance, including:

  1. Multiple roles of nodes, based on different types of hardware capacities to enable dynamic and efficient load management.
  2. Data sharding and scattering, together with separation of data submitting and data persisting nodes to improve storage scalability and data security.

1. Node Roles

Just as allocating housework between housemates increases efficiency in completing menial chores, assigning certain functions to specific nodes allows for higher processing speeds, whilst it also allows the network to scale as additional nodes added are assigned varying functions.

Insolar’s approach involves allocating computing resources (nodes) specific roles, namely by dividing nodes into three categories:

  • Calculation nodes (so called Virtual Nodes) do the processing and are able to introduce changes and new data into storage.
  • Block-building nodes (so called Light Material Nodes) serve as storage for Virtual Nodes, form new blocks and act as short-term (hours or days) storage.
  • Storage nodes (so called Heavy Material Nodes) complete long-term distributed storage with data replication and density controls, plus automated content integrity checks based on Proof-of-Storage techniques.

Separation into these categories is closely coupled with scalability. Increasing the number of Virtual Nodes provides higher CPU throughput; more Light Material Nodes provides higher network throughput; and more Heavy Material Nodes increases storage capacity.

2. Data Sharding and scattering

On the Insolar platform, each object in a shard is independent from other objects and belongs to a Jet — a unit of sharding. Insolar uses two kinds of Jets: a Material Jet, which is close to the understanding of a shardchain; and Virtual Jet, which covers CPU-based processing and data affinity.

Whilst Virtual Nodes carry out all of the CPU-based data processing, they send this processed data to Light Material Nodes which are tasked with splitting data when there is a large load. This process is called data sharding. Additionally, Insolar does time-based distribution of a data object across multiple nodes, so there is no single node in possession of all data of an object. This is called scattering.

Data in Jets

As mentioned above, new Jets appear depending on the splits that occur as a result of the storage and network load. A new Jet forms by branching from the Jet with the large load. Jets keep records of a subset of objects contained by a network and are allocated nodes to store related records.

As well as making up shardchains, Jets are partitioned across time, which makes up storage blocks which we call Jet Drops. Jet Drops are a close equivalent of blocks in other blockchains and, as such, are units of storage.

Again here, the division and distribution of work to allocate separate functions to Jets allows to increase network efficiency. A Material Jet carries out data storage and retrieval for Virtual Jets, just as do Material Nodes for Virtual Nodes in validating transactions. This network structure means storage and processing bottlenecks can be avoided, so the network can cope with larger data loads. Moreover, the structure allows for near-linear scalability of the platform for any number of data objects.

The features of the Insolar blockchain platform were created with enterprise needs as a foundation in order to bring about adoption of the technology. One of the key requirements for business is high throughput for transactions and this is a capability we have demonstrated with the Insolar testnet. Insolar’s network consensus, validation and per transaction consensuses, in addition to role allocation to split workloads are key to the network being able to achieve previously-unseen transaction speeds.

_______

  • Check our Github and leave feedback on the code.

Follow Insolar on social media:

Insolar

Insolar is an open-source enterprise-grade blockchain platform to enable seamless interactions between companies and new growth opportunities powered by distributed trust.

Insolar Team

Written by

Distributed business networks

Insolar

Insolar

Insolar is an open-source enterprise-grade blockchain platform to enable seamless interactions between companies and new growth opportunities powered by distributed trust.

More From Medium

More on Blockchain from Insolar

More on Blockchain from Insolar

XNS文章系列:代币与硬币

Anri Nap
Mar 30 · 2 min read

More on Blockchain from Insolar

More on Blockchain from Insolar

HitBTC上市Insolar XNS!

Anri Nap
Mar 27 · 1 min read

More on Blockchain from Insolar

More on Blockchain from Insolar

XNS Mini-series: Token vs Coin

Insolar Team
Mar 30 · 2 min read
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade