Half-Baked is Always Better than Double-Baked — What is Double-Baking? What Causes Double-Baking?

This article is a continuation of the series Half-Baked is Always Better than Double-Baked. If you haven’t, I recommend reading first Half-Baked is Always Better than Double-Baked — What is at Stake in the Tezos Protocol?, in which we defined the Nothing-at-Stake-Problem (NASP) and explained how the Tezos protocol prevents it. While, the misadventures of Tezos bakers do not seem to cease any time soon, this article explains what it means to double-sign (double-baking and -endorsing) and the possible causes.

This is fine

What Does It Mean To Double-Bake or -Endorse?

Above, we briefly mentioned that PoS protocols are susceptible to the NASP when validating nodes can cause and maintain multiple forks or histories at no cost.

In Tezos, forks are caused when a baker double-bakes or double-endorses. What exactly do we mean by causing forks? Simply put, a baker double-bakes or double-endorses when he/she signs two different blocks or endorsements at the same height. In other words, safety faults can occur when the time for a baker to perform a baking or endorsement comes at a specific block level, the baker that must perform the baking or endorsement operation, signs two versions of the state.

Simplified Example of Double-Baking in Tezos

As the diagram below (attempts to) illustrates, we have a baker named XY who has been assigned the baking slot at level 9000.

How to Double-Bake on Tezos

Under regular circumstances, Baker XY would build one block at level 9000, which is unique due to the data it carries, defined by e.g. the list of transaction it includes for that block [a,b,c,d].

However, due to unfortunate circumstances, this baker has produced another block at the same level that includes a different set of transactions[a,d,e,f]. The baker signed both blocks and following the protocol, the new blocks get broadcasted to the peers in the Tezos network.

Both blocks are first gossiped by different peers, which later on gossip to other peers (in reality, most peers gossip simultaneously to many other peers, instead of to only one at a time). At some point, a node in the network receives both blocks and realises that there are two blocks at level 9000 signed by the same baker. That baker has now evidence and can denounce the safety fault by including the evidence in his/her next baking slot.

Once the evidence is broadcasted (included in a newly produced block) and is verified by other peers, the baker who broadcasted different blocks at level 9000 loses the security deposits of that block 512 XTZ, all the rewards and transaction fees that were locked together with the security deposits.

What Causes Double-Baking and -Endorsing?

Even though there is no evidence, we will assume that all the double-bakings and endorsements in made so far in Tezos were accidental. Note that there is also no proof of what actually happened, but we will explore some possibilities of what could have caused the safety faults. All the possible answers revolve around the type of signing keys.

What Are Baker Signing Keys?

Baker accounts are no different than any other Tezos implicit accounts. They have a public key that starts with TZ1 and a private key. The difference is that the owner of those keys sent a transaction setting the account as a baker and that the account has a balance of at least 10,000 XTZ that can be used as security deposits.

The private key, which is used not only for signing transactions but also for signing blocks and endorsements in the case of bakers, lives somewhere in a specific format depending on the medium and the environment that generated the keys. Depending on how they were generated, there are mainly three types:

  • File-based keys: The baker generated the keys using the tezos-client or popular wallets, such as TezBox Wallet, Cryptonomic’s Galleon Wallet or Magnum Wallet, which offer the feature of exporting the keys of the account into a file. The keys are nothing else than text files that live somewhere on a device. They can be copied, deleted, and modified — just like you could do with any plain text.
  • Cloud HSM: The baker generated the account on a cloud instance using a Cloud Hardware Security Module, normally via an API. The main advantage of Cloud HSMs is that key exfiltration is significantly more challenging than with file-based keys.
  • Physical HSM: When the account is generated using a Ledger Nano S (the only physical HSM supporting baking to our knowledge), the keys live in the hardware wallet and it has to be connected to a full node to be able to produce signatures.

Accidental Double-Baking

Double-baking occurs when there is more than one baking node using the same keys to sign a block or endorsement. The result of the latter is the production of two different blocks or endorsements at the same level. Sometimes, spawning more than one baker with the same keys is, unfortunately, easier than it should: it is as simple as starting the baker without knowing that a previous one is still running. Example scenarios:

  • Lack of coordination among agents in charge of the baker
  • During software updates

Other times it can be due to misconfiguration of a failover baker with the purpose of starting in case the initial baking node fails, to avoid missing baking and endorsement slots. When the failover strategy is misconfigured, it can spawn the backup node before the original one terminates, resulting in both nodes baking or endorsing at the same level.

In summary, with non-malicious or accidental double-baking most of the cases come down to starting two or more instances of the baking node at the same time.

Malicious Double-Baking

Unlike the above, malicious double-baking occurs when an agent starts one or more competing instances of a baking node with the same set of keys intentionally. In order to do so, an adversary must gain access to the keys via remove or physical host compromise, depending on what type of keys are used.

In the case of file-based keys, a malicious agent that gains access to your host could not only spawn conflicting instances of the baker, but also steal the keys. Furthermore, losing control of keys compromises the funds of the account, as they can also sign transactions.

When it comes to cloud HSM keys, it will be significantly harder for the adversary to extract the keys. However, once the host has been compromised, he/she can still spawn conflicting instances of the baker to cause double-baking or sign fund transfer transactions.

Finally, physical HSM keys are vulnerable to the adversary gaining physical access to the hardware. In the case of the Ledger Nano S, the attacker can take the baker down by unplugging the device, causing missing of baking and endorsement slots. Nonetheless, due to the double-signing protection feature of Ledger, the attacker cannot cause double-baking or -endorsing. Even in the event of host compromise, the Ledger would not allow double-signing blocks. While it is not impossible, extracting the keys from a Ledger is in most cases difficult enough to require too many resources to be worth the reward.

Installation and Execution of Faulty Protocol Releases

Following social consensus, all bakers download, install, and execute the releases of Tezos. However, as there is no requirement of auditing or verifying what software each baker runs, the risk of running faulty software persists. By faulty, we refer to versions of Tezos that might include intentional or unintentional bugs or exploits that cause double-baking.

[To be continued in Part III: Half-Baked is Always Better than Double-Baked — How to Prevent Double-Baking?]