Espresso’s HotShot Consensus Protocol is Now Open Source
The Espresso team open sourced the protocol under an MIT license as it presented its latest paper on HotShot and the Espresso Network at the Science of Blockchain Conference
In 2022, we unveiled the HotShot consensus protocol, which we designed at the time to power a proof-of-stake network with high throughput and fast finality for our original shared sequencing solution. Since then, we’ve shipped five testnets based on this novel approach to proof-of-stake consensus, expanded our product mix, and are gearing up for our mainnet launch in the coming months.
Today, we’re excited to make HotShot available for other developers and teams to use under an MIT license. We are committed to collaboration and innovation in the blockchain space and believe HotShot, which was built with Rust and designed to scale to thousands of nodes without sacrificing performance, can serve a wide range of projects seeking a state-of-the-art approach to proof-of-stake consensus, from other blockchains and oracle networks to virtual machines and trusted execution environments.
We announced the news today at the Science of Blockchain Conference, where we presented our latest research paper on HotShot and its implementation in the Espresso Network, which provides participating rollups with fast finality on transaction ordering and data availability, enabling tighter coordination within Ethereum’s fragmented L2 ecosystem.
By making HotShot freely available, we invite developers, researchers, and other builders to join us in refining, advancing, and using this innovative solution for consensus.
Our implementation of HotShot can be found in this GitHub repository, and the docs can be accessed here.
How HotShot Achieves Web2 Speed with Web3 Security
We developed HotShot specifically to power the Espresso Network, a shared source of truth for participating rollups that provide fast finality on transaction ordering and data availability.
The Espresso Network underpins the Espresso Marketplace, which allows rollups to sell temporary transaction-ordering rights to third-party sequencers. The marketplace, which you can read more about here, supports revenue creation, decentralization, and cross-rollup interoperability for the rollups that opt in.
We needed a consensus protocol capable of scaling to thousands of nodes while retaining low latency, high-throughput, and bribery resistance. We took inspiration from an existing leader-based Byzantine fault-tolerant replication protocol called HotStuff, extending it to a decentralized proof-of-stake setting. Credit to Dahlia Malkhi and Kartik Nayak, two members of the Espresso team who were key contributors to the original HotStuff papers.
When network conditions are optimal, HotShot is capable of confirming transactions as fast as the network will allow, which throughout our benchmarking across multiple testnets has been near-instantaneous. This differs from protocols, including Ethereum’s own consensus protocol, Gasper, that set confirmation delays based on worst-case network conditions, or those where transaction finality is only achieved probabilistically.
While Gasper prioritizes liveness under pessimistic conditions (i.e., dynamic availability), resulting in slow confirmation times, HotShot is able to provide fast confirmations by prioritizing performance under optimistic conditions (i.e., optimistic responsiveness).
One factor that gives HotShot its speed is that participating nodes do not execute transactions. This means individual nodes don’t need full access to data for all transactions included in a block in order to vote in consensus, they only need assurances that it is available.
Another factor in HotShot’s performance is its use of a content delivery network (CDN) at the network layer that’s similar to what’s used in Web2 architecture. Achieving low latency requires parties to communicate with each other faster than relying on a peer-to-peer network, which protocols like Gasper use. Using a CDN to enable efficient information routing at low latency enables HotShot to optimistically achieve extremely high throughput, while if faced with suboptimal network conditions can fall back to a high-resilience, gossip-based path with lower throughput. In this sense, HotShot can achieve the best of both worlds: Web2 performance with Web3 security.
Separating consensus from data availability (DA) is a major differentiator of HotShot from other well-known consensus protocols. Tendermint, for example, is an established consensus protocol used by projects such as Cosmos and Polygon. While HotShot is designed to scale to thousands of nodes without a loss in performance, Tendermint is designed to work among tens or hundreds of nodes that not only need to reach consensus, but have full access to all transaction data in order to handle execution and state changes. For more on how HotShot compares to Tendermint, check out ‘What are the Differences between Tendermint and HotShot?’
Various DA solutions can be configured to work with HotShot. To power the Espresso Network, we’ve developed a custom DA solution called Tiramisu that compliments HotShot and benefits from its optimistic responsiveness. Tiramisu provides efficient, Web2-scale DA under optimistic conditions while providing Ethereum-level guarantees under pessimistic conditions.
Benchmarks & Performance
In addition to open sourcing HotShot today, we are also releasing our latest benchmarks and performance.
You can find details of our most recent results and experimental design in our documentation, as well as past benchmarks with prior testnets.
These latest results benchmark HotShot with the addition of the Savoiardi layer to Tiramisu DA. Savoiardi is a bribery-resistant data availability scheme similar to Ethereum’s danksharding proposal. Unlike danksharding or Celestia’s DA solution, Savoiardi does not require data availability sampling (DAS). Instead it relies on a technique called verifiable information dispersal (VID). More can be read about this here.
In our latest benchmarking, the protocol achieves better maximum throughput in large networks than standard consensus protocols where data is sent to all nodes. This is as we expected and an achievement of the design. In these benchmarks, latency is slower than in comparable protocols due to some implementation-specific bottlenecks that we plan to address in future upgrades as we progress toward our mainnet release later this year.
Fair Exchange and the Espresso Sequencing Network
Today, we are presenting a writeup of our work on HotShot at the Science of Blockchain Conference: “The Espresso Sequencing Network: HotShot Consensus, Tiramisu Data-Availability, and Builder-Exchange.”
This paper includes not only details of HotShot and Tiramisu, our DA solution, but also an extension of HotShot that separates the task of proposing new blocks from building the blocks, thus creating an ecosystem of specialized block builders to participate in the Espresso Network.
However, separating the task of block building from proposing brings a new challenge. Builders want assurances their blocks would commit in exchange for revealing their contents, whereas validators/proposers want assurance that the data in committed blocks will be available and fees paid. This is known as the fair exchange problem. Neither party trusts the other, hence the shared sequencing platform should facilitate a fair exchange between builders and the sequencing network.
Currently, the Ethereum ecosystem handles the fair exchange problem by relying on trusted third parties called relays, which pose their own risks.
To address the problem, HotShot has introduced a Builder-Exchange mechanism, which is a combination of two core ingredients: (i) a commit-reveal regime, and (ii) a builder-driven extension. More on how this works can be read in the paper.
The Builder-Exchange mechanism, which enables our aforementioned Espresso Marketplace, is not limited in its use to HotShot and is applicable to other consensus protocols.
Conclusion
With the release of our paper on the HotShot protocol and the open-sourcing of the code under an MIT license, we aim to contribute to the ongoing innovation within the blockchain industry that will lead us towards a more scalable and decentralized future. If you have questions about HotShot, its architecture, the paper, or any of the referenced benchmarks, feel free to join us in our R&D forum.
If you work on a rollup or other infrastructure project and want to explore how Espresso can help you lower operational costs, achieve decentralization, and enable cross-rollup interoperability, please get in touch. We’re currently looking for chains to join our growing list of partners deploying on our upcoming mainnet launch.
