ConcordiumBFT — Live on Mainnet
On Sept. 25th, 2023 ConcordiumBFT, the new consensus protocol, went live on mainnet. This is a tremendous accomplishment as upgrading the consensus engine of a blockchain is a massively complex undertaking that is often compared to changing an aircraft’s engine inflight. Concordium pulled this off without any downtime or glitches!
In this article, we will talk about Concordium’s journey from its previous Nakamoto style consensus with a separate finality layer to its newly launched ConcordiumBFT consensus, while shedding light on the benefits it brings and discussing the need for better benchmarking standards in the blockchain industry.
Blockchain consensus
A blockchain’s consensus protocol is perhaps the most important component of a blockchain network that helps validate and agree on the state of the blockchain. It’s a set of rules and protocols that determine how transactions are verified, added to the blockchain, and how network participants reach agreement on the shared ledger. The consensus algorithm is essential to prevent double-spending and maintain the integrity of the blockchain.
Right from Concordium’s inception, the focus has been on building a highly secure network with fast finalization, which has been facilitated by a strong foundation stemming from a top notch science backed team.
Previous consensus engine — NSC with Finality Layer
Hitherto, Concordium had a Nakamoto-style consensus (NSC) where parties participate in a form of a lottery to win the right to append blocks to the chain. The probability that a certain party wins the lottery depends on how much of the required resource they have, for example, computing power for proof of work systems or stake for PoS systems such as Concordium.
To improve performance and finalization, Concordium had a two-layer approach wherein a finality layer was added on top of a NSC layer to dynamically ‘checkpoint’ the blockchain by using Byzantine agreement to identify and then mark common blocks in the chains of honest users as final.
New Consensus Engine — ConcordiumBFT
To continue providing maximum value to users and the many projects building on it, Concordium has taken its consensus protocol to the next level with the introduction of ConcordiumBFT.
ConcordiumBFT is based on the HotStuff consensus protocol. It is designed to support highly scalable and robust blockchains. HotStuff is based on a type of consensus algorithm known as a “Byzantine fault tolerance” (BFT) algorithm and has been shown to provide faster and more efficient performance compared to other BFT algorithms.
ConcordiumBFT proceeds as a series of rounds, in which blocks are produced and agreed upon. In every round, ConcordiumBFT proceeds as follows: the validator (Bob) elected as leader for the current round produces a new block, appends it to the chain, and distributes it over the network to the other validators. The other validators then verify the validity of the block and sign it. Signatures from validators together holding at least ⅔ relative stake of the finalization committee are aggregated into a quorum certificate, which certifies that block. Once such a quorum certificate is formed, the validator selected for the next round can produce the next block extending the certified block and so on. This process is summarized in Figure 1 below.
If a validator fails to produce a block (or produces an invalid block), the other validators wait for a specified period of time and then sign a timeout message for the current round. If validators with at least ⅔ relative stake sign such a timeout message, a timeout certificate is formed and the next validator can proceed, skipping the current round without a block. Whenever there are two certified blocks in two consecutive rounds, the first of the two blocks is considered final.
Measuring what matters:
In the blockchain industry, it’s common to see vanity numbers being put out that don’t reflect real world conditions and hence don’t affect user experience or performance in any real way.
Max transactions per second (TPS) is a case in point. Often you have chains claiming max TPS of 100 K and above, only to realize that these are theoretical numbers based on lab settings where only the consensus mechanism is tested and not the whole blockchain stack including the networking and execution layers. This is like measuring the horsepower of a car by only measuring the engine’s output and not its ability to turn the wheels.
Furthermore, max TPS is only a relevant number when the network is fully saturated, and hence does not make any difference when one is dealing with a network that is not congested, like is the case with most blockchains.
The more relevant number for blockchain performance is confirmation time. This is the amount of time it takes for a transaction that is sent to the network to be written into a block that cannot be rolled back, i.e. a finalized block.
In ConcordiumBFT, the second last of two blocks in consecutive rounds is considered finalized.
With the current blocktime of 2s, it takes on average 1s for a transaction to be included in the block. This is because it depends on when the transaction is submitted to the network while blocks are being made. Transactions wait in the mempool before being validated into a block. If a block has just been created, then the transaction has to wait a whole 2 seconds for the next block. If on the other hand, it’s submitted just before the next block is made, its waiting time is almost 0 s. Hence, the average waiting time for transactions to be included in a block is 1s.
After this, a new block needs to be appended to the block (another 2s) and get a QC (<0.1s). Once this happens, this block is considered finalized in ConcordiumBFT and cannot be rolled back.
With the waiting times added up we end up with an average confirmation time of ~3s.
Following the above logic, in general, the confirmation time ~ 1.5x blocktime.
Another misused blockchain industry metric is finalization time or time to finality, measured as the time it takes to finalize a block. This, however, is a misleading and irrelevant metric that does not reflect how long it takes a transaction to get finalized, as explained previously.
Hence we have chosen to use the term confirmation time to clearly distinguish transaction finalization time from block finalization time. It is a key metric that reflects the real value that blockchain brings as a trustless decentralized ledger.
In the past, CCDScan displayed Finalization Time, but we do not show confirmation time for the new consensus owning to the fact that it’s deterministic and predictable (~1.5x blocktime).
ConcordiumBFT offers the following benefits:
Consistent and faster block production — the ConcordiumBFT consensus ensures that blocks are produced more consistently and, on average, a lot faster. Compared to the current Nakamoto style consensus, ConcordiumBFT is at least 5x faster in block times, currently around 2s.
We note that ConcordiumBFT is capable of achieving even faster (sub-second) blocktimes. It’s currently set to 2s only to minimize empty blocks and disk space wastage. With growing on-chain transaction volumes, the blocktime can be tuned to be sub-second.
Consistent and fast finalization — as a consequence of more consistent and faster block speed, the confirmation time for transactions is also consistent and faster providing a best in class user experience. The confirmation time for transactions is around 3s with the current blocktime of 2s.
Higher TPS — fast block speed and finalization ensure that the max TPS increases by 5x, as well, supporting even the most advanced use cases where high bandwidth is important. We are currently seeing 2000 TPS in internal tests.
Improved validator management: In ConcordiumBFT, the queue of upcoming validators for the current epoch (1 hour slot) is know, hence validators can know when their turn is coming up, enabling validators to better manage their nodes. They can, for example, make sure that they are running the latest node software and are not offline when their turn comes up. This way, they can be sure to not miss out on any block rewards.
Transparency over validator performance: To support the long term growth of the Concordium blockchain, the quality of the nodes in the network is important. Current data suggest that well-run nodes are able to attract more delegators. The upgrade increases the transparency of validator performance by displaying this information in CCDScan. Delegators then have better metrics when choosing which validator to delegate their stake to, incentivizing validators who are running their nodes poorly to step up their game because they know that otherwise they will not get any delegation.
Difference between old and new consensus protocols:
Importance of benchmarking — A side note
The multitude of blockchain options presents a dilemma for developers and businesses aiming to incorporate blockchain solutions. It becomes challenging to discern which blockchain is the best fit for a specific use case because each blockchain highlights its own performance attributes, unique features, and strengths. The benchmark data they publish often appear to favor the blockchain they are promoting and are generally not subject to independent verification. Additionally, these benchmarks lack uniformity, with most blockchains measuring throughput in transactions per second (TPS) without a standardized definition of what constitutes a “transaction” and what the runtime environment looks like. Consequently, these numbers are incomparable and offer little practical guidance for decision-making. Presently, the in-house benchmarking provided by blockchain organizations primarily serves marketing purposes.
This is similar to a car company using unstandardized benchmarks. Take the Tesla Model S Plaid, for example, which advertises a 0–60 mph acceleration in 1.99s. What they don’t clearly state though is that this does not include the first foot of the acceleration where static friction is overcome, skewing the results in their favor.
The blockchain industry can take inspiration from the database community which has a robust and mature set of methods for benchmarking. The Transaction Processing Performance Council (TPC) offers a variety of database benchmarks, including standardized ones for transaction processing, decision support, and virtualization. These benchmarks yield results that can be compared across different systems, assisting external parties in making informed choices when selecting the most suitable system for a particular task. The standardization of such benchmarks within the blockchain industry is crucial to promote the adoption of blockchain technology.
Conclusion
With the new consensus engine the Concordium blockchain is truly amongst the few top chains in terms of performance and security. Not once in its mainnet lifetime has the network had a disruption, a true testament to its robustness, and now it has proven itself on the speed and scalability front as well.
PS. To learn more about ConcordiumBFT, watch this video.
