The Design Philosophy of Hyperledger Fabric

How does Fabric improve the blockchain? What are its features?

Fabric was born in 2015 and is one of Hyperledger’s earliest projects. It is mainly developed and open-sourced by IBM. Fabric can be said to be a distrubuted ledger that is designed exclusively for enterprise and is known for its good performance and smart contracts, called “chaincode”, that can support general programming languages such as Golang and Nodejs. However, on the other hand, its complex architecture, documents that full of obscure technical terms, and a design that is completely different from the mainstream blockchain also make developers daunted and cannot see the wood for the trees.

The goal of this article is to make you see and the trees and the wood. The overall operating logic and complex architecture of Fabric can be more comprehensive as long as the design philosophy of Fabric is clear. In this article, I will start from “Hyperledger Fabric: A Distributed Operating System for Permissioned Blockchains” published by IBM researchers to discuss:

• Limitations of mainstream blockchain technology
• Design philosophy of Fabric
• Transaction workflow of Fabric
• Unique features of Fabric

The blockchain technology known to the general public mostly refers to the technology originated in Bitcoin/Ethereum. In order to distinguish it from the technologies to be mentioned below, I will temporarily refer to the mainstream blockchain technology as “Nakamoto Blockchain”.

Limitations of Nakamoto Blockchain in Business Collaboration

Technical Overview of Nakamoto Blockchain

The most important feature of Nakamoto blockchain is that it is “Permissionless”. Therefore, it has a sybil-control mechanism and a unique consensus model to keep its trust assumption, as well as fairness and incentives.

However, the consensus model that makes Nakamoto blockchain successful is also its biggest performance bottleneck. Why is it?

“Order-Execute” Model and Active Replication

The consensus model of Nakamoto Blockchain is called the “order-execute” model. As the name implies, it first reaches a consensus on the order of transactions and then the transactions are executed by all nodes in order. At the same time, each node participating in mining must “execute” each transaction that is about to be wrapped in the block by itself before “ordering”, and discard the invalid transaction; all nodes “re-execute” the transactions in sequence in order to validate the result after receiving the block. The local ledger is updated after the re-execution.

This state replication mode is also known as “active replication”. In this mode, the node can only update its own state through the repeated execution results of the computation process. You might notice: no matter how resource-consuming for that transaction is, it must be repeatedly executed by each node, which makes Nakamoto blockchain inefficient.

The Trade-off between “Permissionless” and “Performant”

We should be asking: why does Nakamoto blockchain use inefficient “active replication”? Everything dates back to the “permissionless” part of the Nakamoto blockchain. Without permission, every node could be the block producer, which is assumed to be a Byzantine node. To maintain the trust assumption of Nakamoto blockchain, it is necessary to validate every transaction by re-executing it.

Other Limitations

In addition to performance issues, the “order-execute” model has some other limitations:

• Sequential Execution: Since each node relies on the “sequential execution” of transactions to update its local state, scalability is also limited, and it is difficult to use parallel processing and other methods to improve.
• Deterministic Code: In addition to ensuring the order of transactions, it is also necessary to ensure that the execution of transactions is deterministic. This also limits the ability of smart contracts to be developed using general languages such as Golang and Nodejs, because general languages mostly contain non-deterministic features and internal implementation. For this reason, new language must be designed, such as Ethereum’s smart contract language Solidity and its virtual machine EVM.
• Execution Confidentiality: Since the transaction must be executed by “all” nodes, all the states saved in the Nakamoto blockchain must be public and accessible so that each node can read/write these states. This sacrifice transaction confidentiality because any node can know the logic of any smart contract and its internal state.

Not Suitable for Business Collaboration

Efficiency bottlenecks, lack of privacy, and high development costs all bring the difficulty of using blockchain technology in business collaboration. In the public network, it is acceptable to sacrifice performance for “permissionless,” but it may not be appropriate in business collaboration.

However, the network environment between enterprises is usually not like a public network that can be used by anyone permissionlessly, that is to say: The “order-execute” model that originally designed for permissionless access might be further improved. Therefore, many designs for improving Nakamoto blockchain have emerged one after another. Among the many ideas, the most well-known model is designed by IBM researchers, which later became the core of the Fabric architecture.

So what is this new model?

New Model: Design a Distributed Ledger from a New Perspective

Overview of New Model

As mentioned above, business collaboration network is the opposite of the public network: It is relatively limited. In addition to fixed participants and real-name authentication, they also have access control, which is inherently a “permissioned” environment. A permissioned network has a relatively loose trust model, which means that the probability of participants not doing evil is larger, which gives the consensus model greater design flexibility.

The design proposed by IBM researchers can be condensed into one sentence: “executed by minority, validated by majority”. This is quite intuitive: in a permissioned network, as long as the “part” of the nodes have to execute transactions under the minimum conditions sufficient to ensure security, most nodes do not need to execute repeatedly and only require a lightweight validation before their state is updated. So how do you ensure security with only a few nodes? This can be done by a “consensus” between a small number of nodes that executes the transactions.

“Execute-Order-Validate” Model and Hybrid Replication

Based on the above ideas, IBM researchers designed the “execute-order-validate” model. First, the transaction will be passed to some pre-designated nodes to “execute”. A consensus can be reached if the execution results are consistent. Transactions that have not reached a consensus will be discarded. Next, transactions from different nodes that have reached a consensus will be passed to a specific service to be “ordered” and packaged into blocks; finally, this specific service propagates the block to all nodes. The node receiving the block only needs to perform a lightweight “validation” on the transaction, and can update local state if valid.

This model shows the spirit of “executed by minority, validated by majority”. The first benefit is that it allows most nodes to save the work of repeating transactions and can balance performance and safety in a “permissioned” network. In addition, it has some good properties, which will be addressed in detail below.

Compared with “active replication”, directly updating the state without computed again is called “passive replication”, and the “execution-order-verification” model is neither “active replication” nor “passive replication”-since there is still “ordering” and “validation” work, but “hybrid replication” which is a combination of above two. The “active” part is: the consensus of the “execution” result reached by a few nodes can represent the validity of the transaction, because only the state of transaction that has consensus will eventually be updated by all nodes; the “passive” part is: most of the nodes do not need to execute the transaction, and only need a lightweight validation to update its state.

Next, let’s see how this model works.

Overview of Fabric

From the above design philosophy, we can know that in Fabric, “execution”, “ordering”, and “validation” will be passed to nodes with different roles — This is different from the design of Nakamoto blockchain in which all nodes have do the same work.

Fabric consists of three main roles: Client, Peer and Ordering Service. They form a “channel” and share the same state, only the nodes of the channel can access the state of the ledger. Each Fabric node can simultaneously construct different channels and each channel is an independent blockchain network with its own ledger.

Next, let’s take a look at the main functions of these roles:

Client

The client is the top-level application and is the main interface for users to interact with Fabric through the software development kit (SDK) provided by Fabric.

Peer

The peer is the most important role in the Fabric channel. It is responsible for the “execution” and “validation” of transactions, as well as storing the blockchain data and ledger state.

According to the “execute-order-validate” model, only a small number of designated nodes need to “execute” transactions, and other nodes only need to “validate”. Among them, the node that executes is called “Endorsing Peer”; the node that validates is called “Committing Peer”.

The structure of each peer node is the same, and each task is completed by the corresponding module. Let’s take a look at the most important modules:

• Endorser: Implement the “execution” of the transaction. It uses a Docker container to support chaincode developed in different general languages. The chaincode and endorsement policy have to be deployed by the client in advance.
• Committer: Implement the “validation” of the transaction and updating of the state. It also contains some configuration files and system chaincode.
• Membership Service Provider (MSP): MSP is the key module to make the network “permissioned”: it is responsible for the identity and permission control of the node. Fabric uses X.509 certificates for identification, which is compatible with existing identity management systems so that Fabric works well with existing public-key infrastructure (PKI). On the other hand, MSP is only responsible for managing, not for issuing certificates. So who can issue certificates? The Certificate Authority (CA) does, whether it is a commercial CA or Fabric-CA, which is native to Fabric.

Ordering Service

Ordering service makes Fabric unique. It provides a consistent order of the transactions and produces blocks for each channel.

This service can be a single node or a cluster of multiple nodes. Compared with stateful “peer” mentioned above, the ordering service is stateless since it doesn’t has to neither “execute” nor “validate”. This brings a good feature to Fabric: since there is no need to handle with state synchronization, ordering service can scale-out more easily.

I mentioned two types of consensus: the endorsement policy and the ordering service. The difference between these two consensuses will also be detailed below.

Transaction Workflow of Fabric

What process does a transaction go through from initiation to writing in the ledger? From the “execute-order-validate” model, the transaction will go through three stages: “execution”, “ordering” and “validation”. The overall workflow is shown in the sequence diagram above. According to the labels in the above figure, the “execution” corresponds to step 1 and step 2; the “ordering” corresponds to step 3 and step 4; the “validation” corresponds to the paragraph described in the bottom text.

Let’s take a look at the transaction workflow more detailly:

1. Execute

• First, the client sends a transaction proposal with digital signature to one or more endorsing peers. The content of the transaction proposal includes ChaincodeID, parameters, etc. When deploying a chaincode, you need to attach the corresponding endorsing policy, which specifies the endorsing peers and the threshold of reaching a consensus, for example: 3 of 5 endorsing peers need to have the same execution result.
• Next, the endorsing peers start executing the transaction to simulate the results. The chaincode will be deployed on the peers in advance and will be executed in the Docker container. The state of a node is represented by a versioned key-value pair. The state that needs to be read before each execution is called “read set”; the new state that changes after execution is called “write set”. Likewise, they both have versions.
• The endorsing peer wraps the result of the execution into an “endorsement” and returns it to the client. Its content includes read set, write set, and some meta data such as TransactionID, EndorserID, signature of the endorsing peer, etc.
• The client collects “endorsements” from different endorsing peers until the consensus threshold specified by the endorsement policy is met, that is to say: a certain number of endorsing peers must have the same execution result — that is, the same read set and write set.

2. Order

• If the endorsing policy is satisfied, the client then wrap the execution result into a transaction and sends it to the ordering service; otherwise, it discards the transaction. The transaction contains the data payload, metadata, and endorsements from each endorsing peer.
• The ordering service will “order” all the transactions received within a period of time and produce a block.
• Next, the ordering service sends the block with all transactions to all committing nodes. Here, the ordering service as the central coordinator has a more efficient message propagation than the gossip protocol, and the communication complexity is reduced from O(n²) to O(n). Although the bandwidth of the ordering service will also be a bottleneck, it can reduce the communication overhead via options such as using a larger cluster.

3. Validate

• Finally, the committing node will validate the transactions after receiving the block — Since the entire process does not need to “execute” the transaction, it is very fast and lightweight. Validation can be subdivided into three steps: first, the endorsing policy evaluation is to check whether the endorsement attached to the transaction really meets the endorsing policy; then, the read/write conflict check ensures the version of the “read set” is consistent; if there is no problem with the above, committing peers update the ledger according to the write set.
• Notice that the committing node needs to track whether the block contains invalid transaction. If the transaction is invalid, it is directly discarded. In Fabirc, an unique countermeasure can be used if Byzantine faults are detected: Put the endorsing peer directly in the blocklist.

A Feature that Makes Fabric more Flexible: Consensus Decoupling

Although the goal of the “execute-order-validate” model is to solve the performance issue of Nakamoto blockchain, its unique architecture also brings some additional features, especially the decoupling of consensus. Fabric has two levels of consensus: application consensus based on endorsement policy, and protocol consensus based on ordering service.

• Application Consensus: For developers, they can specify different endorsing policies for each chaincode. For example: for applications that require higher trust, the consensus threshold can be set to be higher; otherwise, the consensus threshold can be set to be lower. This gives more flexibility in design. In addition, the application consensus also enables non-deterministic transactions since it only needs a few endorsing peers to execute.
• Protocol Consensus: For system maintainers, they can change the consensus algorithm for ordering services according to actual needs. For example, in inter-organizational collaboration, you can choose a Byzantine fault tolerant (BFT) algorithm, such as PBFT; in collaboration within the same organization, you can use a crash fault tolerant (CFT) consensus algorithm, such as Raft.

Summary

In summary, we can see that Fabric tries to improve Nakamoto blockchain through the new consensus model. In addition to being more efficient, it also has better scalability, modular consensus, smart contracts supporting general programming languages and identity management systems that are easy to integrate.

Finally, let’s recall again: Nakamoto blockchain and Fabric have completely different design goals. The safety of Fabric depends on the permissioned settings and the off-chain trust between enterprises. It is not practical to use Fabric in non-enterprise and public scenario since it does not have sybil-control mechanisms.