Towards a Unified Metric for Performance Evaluation of Proof-of-Stake Blockchains

Sergey Gorbunov and Silvio Micali

Blockchain systems need to scale to process thousands of transactions per second, while remaining decentralized and secure against powerful attack vectors. However, decentralization, security and performance always play a 3-way tug of war and trade-offs between these properties are typically made. These properties are the core pillars of a blockchain system! Simultaneously achieving high-degrees of all three properties for any system is very challenging.

Common Metric Needed

One of the problems we see with the ecosystem is a lack of a common framework for analyzing performance of blockchain systems. Different projects rely on totally different measures, making meaningful comparison essentially impossible. Often hundreds, thousands, or millions of transactions per second are claimed without clarifying the underlying assumptions or settings.

Analysis Vectors

It is important to realize that there are many factors that affect blockchain performace. There are two important performance metrics:

a) Throughput: measured in transactions per second (TPS), and

b) Finality times: the amount of time necessary to ensure that transactions and blocks will (no longer) change.

In this blog post, we focus on throughput and will follow up in a separate discussion on finality times. The following set of dimensions affects blockchain performance:

1. Permissioned vs Permissionless Mode.

A blockchain system may operate in a permissionless mode (where anyone can join the system), or a permissioned mode (where a user must be granted a special approval to join and perform important functionalities for the system).

Permissioned systems are often easier to build and to scale, since one does not need to worry about a variety of attack vectors (e.g., sybil attacks) that are possible in permissionless systems.

2. Direct Participation, Delegation, or Staking Pools.

Individual users can participate directly in the consensus protocol, or delegate their “voting rights” to pool of delegates. Alternatively, only a specially selected pool of users can participate in the consensus (e.g., in bonded proof-of-stake systems).

In an ideal decentralized blockchain, every user is able to participate in the consensus protocol. Delegating his/her participation stake to a smaller pool of delegates (only if he/she wishes to do so) should be a user’s right, not a prerequisite for the system to run. Delegators may enforce additional requirements on the delegates. For instance, delegates may be required to maintain 99.9% availability of their nodes or commit to higher bandwidth allocations. (And it should be possible to verify their fullment of these obligations!) Similarly, in bonded proof-of-stake systems, the specially selected users may be required to allocate more resources (but are in turn more vulnerable to additional attack vectors which must be evaluated). Whether users participate directly in the protocol, or delegate their stake is a choice or enforced by the system. Both the percentage of the money delegate, and the total number of delegates should be reported.

3. Propagation Network Type.

It’s important to understand whether a peer-to-peer propagation network, or a untrusted relay propagation (or a content delivery) network is used.

In a classical blockchain instantiation, transactions and consensus messages propagate in a peer-to-peer network to every node. This implies that a node must propagate every transaction to all of its peers. Alternatively, a relay propagation network may be used to speed up the propagation of transactions and messages of the underlying consensus protocol. Propagation networks are designed to optimize for performance, but they also must resist a variety of additional attack vectors (e.g., denial-of-service) and not introduce attacks of their own (e.g., a centralized relay should not be able to censor users or fork the blockchain)

4. Transactions Visibility.

In a standard blockchain instantiation, all transactions are recorded on the blockchain and available for anyone to inspect. Various techniques (e.g., sharding, lightning networks) can be deployed to shard the blockchain storage, or perform additional transactions off-chain.

Often, these techniques are built as a layer above the underlying consensus protocol. In analyzing performance, therefore one has to specify how many transactions are explicitly tracked on the blockchain and how many can be handled off-chain. Indeed, some transactions are less meaningful, or no meaningful at all, if handled off-chain.

5. Bandwidth Allocations.

Nodes that allocate more bandwidth can propagate blocks of transactions faster to their peers. When making performance analysis of a blockchain system, it is important to make accurate assumptions on the bandwidth allocations of all participating nodes.

6. Envisaged Security Model.

There is always a trade-off between security and performance. It is very easy to be fast and insecure. Indeed, everything in a blockchain system can be attacked! A blockchain should be secure against attacks by Byzantine nodes, external attacks on nodes participating in the consensus (e.g., denial-of-service), attacks on incentives mechanisms, network attacks, etc.

7. Compression Mechanisms.

The quality of compression mechanisms used for encoding transactions and blocks greatly affects overall performance of the system. Succinct encoding schemes can help minimize the required bandwidth from the nodes. Systems must clearly indicate the assumptions on the underlying transactions sizes, or any additional messages that propagate through the network.

Sample Theoretical Analysis

As an example, let us demonstrate a theoretical calculation on the transaction rate in a specific setting.

• Permisionless mode
• Every user participates directly in the consensus protocol.
• All transactions and blocks propagate to every node in the system via a peer-to-peer network. Every node is connected to 8 peers.
• All transactions are confirmed and posted on the blockchain.
• Each node allocates 56 Mb/s of upload speed, and 112 Mb/s of download speed (that is, the median bandwidth allocated by Bitcoin participants).
• Each transaction is encoded in 200 bytes, and a block is confirmed each second.
• There is no overhead for the consensus protocol to agree on a block of transactions.

Given the above parameters, an absolute maximum throughput that a blockchain system can support is (56Mb/s)/(200 bytes * 8 peers) = 4375 TPS.

It is important to highlight that the above analysis ignores the standard network properties such as packet loss, propagation or transmission delays, number of users in the system, retransmission, etc, and provides an upper bound on the transaction rate given the above parameters.

Toward Standardization

We believe it is important to start a discussion on standardization of both theoretical and empirical analysis of the blockchain systems.

Algorand is committed to helping the community navigate through the variety of projects and clarify some frequent confusions. We encourage the community to consider specifying the list of parameters above when evaluating or explaining a new blockchain project.

We also encourage the community to comment on this blog, suggesting additional parameters, and will update the list to reflect the feedback.