Design of the Fetch.AI Scalable Ledger

Over the past few years, scalability has been the topic at the forefront of decentralised ledger technology. During this time, many projects have made claims of tens of thousands and some up to even millions of transactions per second (TPS). On established platforms, such as Ethereum, the typical demand for transactions is well below the lower end of this range. The novel ledger designs have thus been operating under test conditions. It remains to be seen how successful they will be in supporting huge transaction volumes in practice. Unlike test conditions, this will need security and low costs to also be maintained. The prize for solving these challenges is a slice of the multi-trillion dollar future market for blockchain technology. One of these markets is the machine-to-machine economy, which is the target of the system under development at Fetch.AI.

In the decentralised world envisaged by Fetch, every device on the Internet is equipped with an Autonomous Economic Agent (AEA). This is piece of software that conducts communication and engages in financial exchange. The Fetch ecosystem enables agents to collaborate with each other to find solutions from the bottom up. This technology could revolutionise supply-chains and energy markets, to name a couple of examples. The requirements of these industries place certain demands on the ledger. It must scale to millions of transactions while still delivering rapid confirmation times. Another key need is an expressive and low cost smart contract system. In this article we provide an accessible introduction to the scalable ledger. Our technical white paper contains further details.

The architecture of a typical blockchain is shown in Figure 1.

Figure 1. In a typical blockchain, the chain is extended one block at a time by nodes in the decentralised network. A proposed block is broadcast to other nodes. These decide whether to add it to their local representation of the blockchain. If the nodes local representations of the blockchain become identical then this constitutes a global state.

This contains several points that could act as potential bottlenecks:

· Block Synchronisation Latency in broadcasting mined blocks across the network. Increasing the rate of block production or the block size might increase throughput but at the cost of reduced security.

· Memory/Storage Maintaining a complete record of the blockchain becomes expensive for participating nodes.

· Contract Execution The time required to execute transactions could act as a bottleneck. This applies to ledgers with complex smart contract languages and large state databases.

A simple solution to the scalability issue is to simply duplicate the blockchain. This divides the network traffic and contract execution between the independent chains. It’s possible to repeat this process many times to scale the network to an infinite number of TPS. (See Figure 2). Although, this leads to the problem of how to move transactions between the different chains.

Figure 2. A doubling of the transaction throughput can be achieved by duplicating the blockchain.

One approach is to use another layer to mediate communications between the different chains. This often involves specifying a reference chain to resolve conflicts between subsidiary chains. These subsidiary chains are often described as side-chains. The downside is that these approaches can lead to other bottlenecks or security vulnerabilities. The Fetch approach to resolving this issue instead associates each sub-network with a structure known as a resource lane. Nodes that mint blocks, also known as block producers, sit at intersections of the independent sub-networks. (In Figure 2, these nodes are members of both blockchains and are represented by concentric green and purple circles). A node that has been nominated to be the producer for the next block has access to a set of transactions from several resource lanes, as shown in Figure 3.

Figure 3. Resource lanes and block architectures. Each horizontal line represents a single resource lane. (In this example there a five lanes, each represented by a different colour and city). Vertically oriented pale blue blocks represent transactions. These contain coloured circles that denote the resource lanes used in the transaction. In this example, the left-most transaction involves resources in lanes 1 and 5. These lanes correspond to Tokyo and Mumbai. The vertical black line represents the final valid slice in the block. In this example, the block producer could earn a higher fee by moving the transaction in the fifth slice inside the block. The arrow shows the transaction’s move to an empty slot in the second slice.

The blocks in the Fetch ledger are divided into a fixed number of slices that can each contain many transactions. The transactions involving a single lane are equivalent to within-chain transactions. Cross-chain transactions affect multiple resource lanes. To be valid, all of the transactions within a slice cannot affect overlapping resources. This is the key property that enables the transactions within the block slice to be safely executed in parallel. For example, it’s possible to execute a transaction that only involves resources from lane 1 at the same time as a transaction that only involves resources from lane 3. This is possible as the two transactions involve a non-overlapping set of wallet addresses and state variables.

An analogy that can help illustrate this concept is to imagine that there is a transaction, let’s call it A, between two users in London. At the same time a second transaction, B, is sent between two users in New York. The geographical information alone is enough to ensure that these transactions cannot be in conflict. It’s possible that a different transaction, C, between two London users might also not be in conflict with A. (One type of conflict might be spending money that has already left an account in transaction A). Although, checking whether this is the case requires extra computational effort. This problem becomes much more acute when more complex transactions such as smart contracts are executed. The resource lane architecture allows these sequential checking steps to be performed in parallel. And this is the property that increases the ledger’s execution throughput.

From the perspective of the block producer, the aim is to pack as many transactions as possible into the block of a specified size. This allows them to maximise the fees that they receive (Figure 4). This process is analogous to the one of the key innovations in the design of modern CPU architectures. In these processors, non-conflicting operations are arranged so that they can be executed within the smallest number of clock cycles. In the ledger, higher performance arises from the incentive for the block producer to increase the fees they receive. After packing of the transactions, they are cryptographically secured with a Merkle tree. The block is then broadcast across the network.

Figure 4. By packing as many transactions as possible into the block, the producer maximises the fee they receive. A side-effect of this process is that this also increases the number of transactions that can be executed in parallel.

The design of the Fetch ledger enables removal of all the potential bottlenecks on throughput. This is achieved by increasing the number of resource lanes in operation. This is similar to the naive approach of duplicating independent chains. Though it also solves the problem of how to enable secure cross-chain transactions. The design also allows the number of lanes to increase or decrease to meet changes in demand. The high throughput and strict ordering of transactions make the ledger an effective platform for delivering cheaper and more expressive smart contracts. A final advantage of the design is that it represents as a sharding scheme. This allows deployment of light nodes that keep track of only some of the resource lanes in the blockchain.