Different types of evidence in Tendermint
In this article, you will learn what’s happening when the light client encounters two different headers at the same height, how these headers are processed and eventually committed on-chain as evidence of misbehavior.
Part 1. The light client
If you don’t know what the light client is, please read our previous article.
In short, the light client is a process that only verifies a particular state machine’s consensus, without executing the transactions. This allows it to be used in mobile wallets or other low-powered devices.
The light client connects to a set of full nodes and verifies the new headers can be trusted. Most of the communication is happening with just one node, called a primary. Other nodes are called witnesses. We’ll talk about them later.
When a new header is committed on-chain, the light client downloads it from the primary and verifies either sequentially (2/3+ of the voting power from the previous trusted header signed this header) OR skipping (1/3+ of the voting power from the latest trusted header signed the new header). Note in sequential verification headers are serial (e.g., 1,2,3 …). In skipping verification, headers are not serial, and most of the intermediate headers will be skipped (e.g., 1, 10, 2345). Skipping verification is the default method because it’s faster than sequential verification in most cases. These methods provide different security guarantees, which we’ll talk about later.
The new header is also cross-checked with witnesses to confirm all nodes have the same state (~ witness the header).
Unfortunately, we live in a land of byzantine actors, where bad things can happen, and we have to protect ourselves from them. The primary can try to fool the light client to get it to trust an incorrect state.
When the light client gets the header ‘B (from any witness), which hash does not match the one received from the primary (‘A), it tries to verify ‘B (again, sequentially or skipping) against the latest trusted state.
** header’s hash is a Merkle root of the tree built from it’s fields
If ‘B successfully verifies, then we have a fork 🍴
There are two possible scenarios: either somebody (primary and/or witness) is trying to fool the light client (“light fork”) or there’s a legitimate fork on the main chain (“full fork”).
Part 2. How to fool the light client. Step-by-step guide 🦹
2.1 Skipping verification
In order to create a light fork, 1/3+ of the voting power must sign a header with an invalid (malicious) application state. Since the light client does not execute transactions, it can’t check if the application state is valid or not. Only full nodes can do so.
There are five fields in the Header, which can be changed arbitrarily: ValidatorsHash, NextValidatorsHash, AppHash, ConsensusHash, LastResultsHash. Therefore, if a group of malicious validators successfully pulls this attack, they can fool the light client into trusting the invalid application state, validator set, consensus parameters and/or results of executing the previous block.
We call this lunatic validator attack since validators must be lunatic for voting on something as ridiculous as such blocks.
If all witnesses of the light client respond with the same invalid header (meaning they are malicious as well), then the light client will mark it as trusted. That’s why it’s super important to have multiple geographically distributed witnesses, ideally all run by different entities.
2.2 Sequential verification
With sequential verification, malicious validators need more power: 2/3+ to perform the lunatic validator attack. With such power, though, they could perform an actual fork on the main chain and trick both full nodes and light clients into trusting the invalid header(s).
In the next part, we’ll talk about other types of attacks targeting both full nodes and light clients.
Part 3. How to fool a full node. Step-by-step guide 🦹♂️
Now, you might think with the full nodes, malicious validators will need at least 2/3+ to create a fork because they provide higher security guarantees compared to light clients.
And it’s quite right, they will need 2/3+ of the voting power to change the application state we’ve talked about earlier arbitrarily. However, even with 1/3+ of the voting power, they can still create a fork.
Please note that all the listed attacks apply to the light client as well. I.e., if someone can fool a full node, he can fool the light client too.
3.1 Equivocation
The first type of attack is called equivocation or double voting. It’s when a validator (or a group of such) votes for two different blocks within a single round. I want to remind you that Tendermint consensus operates within heights, where there can be from 1 to N (N <= MaxInt32) number of rounds per height. Voting for more than one block within the same round is strictly prohibited.
3.2 Flip-flopping Amnesia
The second type of attack is a bit more complicated and warrants the explanation of locking rules in Tendermint consensus.
Suppose the header ‘X was proposed in the first round (0), but wasn’t committed. The malicious validator locked on the header ‘X because they’ve seen the 2/3+ prevotes for it and precommitted that block. According to Tendermint consensus rules, the validator cannot vote for a different value (~header) without seeing 2/3+ votes for it in the later rounds (1,2,3 …). In the above example, the malicious validator votes for the header ‘Z even though it’s locked on the header ‘X. We call this attack Flip-flopping Amnesia.
3.3 Bonus way to fool the light client
If instead of going forward, the malicious goes backward in time and signs a header ‘X, which didn’t have 2/3+ at the time, but now does, it can fool the light client. The attack is called Back to the past Amnesia.
Even though we’ve explained each type of attack separately, in reality, a cabal group of validators may conduct an attack, where different validators perform different types of attacks (e.g., one commits equivocation, the rest — amnesia).
Part 4. Light Client Attack Detection
When the light client encounters two different headers (e.g. ‘C the from primary, ‘D from one of the witnesses), it tries to verify ‘D. It may request intermediate headers in the process (‘B in the picture below).
If ‘D successfully verifies, the light client forms LightClientAttackEvidence. This contains the first header from the primary that diverges from the trace of headers provided by the witness during verification (i.e. if ‘A is not equal to ‘B then this is ‘A).
In the above diagram, the light client would send ‘A along with ‘C to the witness and ‘B along with ‘D to primary. It sends the evidence to both nodes because at this point it can’t say which one is faulty.
Upon receiving LightClientAttackEvidence, the full node verifies it and determines the particular kind of attack by running the following algorithm:
The full node checks if “previous block dependant fields” (ValidatorsHash, NextValidatorsHash, AppHash, ConsensusHash, LastResultsHash) are derived deterministically from executing the previous block. If not, all validators are considered to be a lunatic for voting on the wrong state.
Note multiple blocks for the same height may be valid since they can contain different previous block independent fields (DataHash, ProposerAddress, etc.). Still, they must all contain the same “previous block dependent fields”.
Tendermint consensus operates within heights, where there can be from 1 to N rounds per height.
If validators vote for a different header within the same round, they have committed an equivocation (or double signing).
Otherwise, they performed the amnesia attack.
The problem with the amnesia attack is that it’s not immediately provable. To prove some validator performed an amnesia attack, we need to gather all the votes for all rounds and check whenever the un/locking rules were violated. The algorithm for that is currently in development. Stay tuned for updates!
When LightClientAttackEvidence is committed on-chain, Tendermint sends a list of malicious validators along with the attack type to the ABCI application (BeginBlock). It’s up to application developers how much to slash a validator’s stake then.
Outro
For a long time, Tendermint only had one type of evidence — Equivocation. Now that Tendermint has the full fledged light client implementation, we must ensure it’s secure against all kinds of attacks, even if some of them require large amounts of voting power and potentially, coordination of multiple malicious parties. With the IBC approaching 1.0 release, it’s especially important to make sure that the light client notices any suspicious behavior and prevents invalid token transfers between Cosmos zones.
Appendix A. How secure the light client is?
As of Oct. 13 (v0.34.0), amnesia attacks do not lead to punishment. As explained in Detailed Walkthrough Of Performing a Light Client Amnesia Attack, which I highly recommend you to read, “attackers need to rely on the assistance of some form of a network partition or on the nature of the sporadic voting to conjure their desired environment”. This is to say it’s hard to execute, but since it requires very little efforts and the malicious validators won’t be slashed, amnesia attacks pose a potential risk to the network.
Appendix B. What about full node forks?
As of Oct. 13 (v0.34.0), the full nodes running Tendermint can only detect equivocation attacks. There’s an ongoing research, which we hope will lead to full nodes being able to detect a full fork and punish those responsible for it. You can follow it here.
I would like to thank Tess Rinearson, Federico Kunze, Erik Grinaker and Callum Waters for reviewing this article ❤️ and Josh Lee for the cover image 🏞️
