Playbook For Cosmos Validators: Key Management Choices
Hold your keys!
This is Part II of a series of posts dedicated to exploring the topic of staking and validation as Cosmos Validator (Part I here). With launch of Cosmos Game of Stakes, we will witness the biggest adversarial BFT distributed network. The bravest participants will use Hardware Security Modules (HSMs), namely YubiHSM2 to validate in this network.
Part I. Anti-DDoS solution for validators
Part III. Setup a Relay network between validators
Table of Contents :
Part I: Account Key Management
Part II: Validator Key Management
Part III: HSM Solutions
Keys in Cosmos Network
There are two types of keys in Cosmos network: Account key and validator key. Let me explain their differences to you.
Account Key
Blockchain keeps track of balances on every account/address. Tokens on a given account are used with the help of a private key — a combination of letters and numbers which are impossible to guess. Anyone who holds the private key of an account will have the right to transfer the balance or delegate them to earn rewards.
Each validator should have some initial tokens on their account, then claim to become a validator. As a delegator, the account key is needed for complete staking transactions:delegate, redelegate, unbond and vote . It’s recomand that validators and delegators could have different accounts for receiving rewards and bonded funds.
Losing account keys will be a disaster for any Atom holders. It’s wise to keep your private key of large balance on an offline device. In this way, hardware wallet seems to be the best choice. Ledger Nano S is one of the most popular hardware wallets in the market. I have written about how to install Cosmos App on your Nano S in this article a few month ago.
Now, Cosmos app is now available in Ledger app store in developer/testnet mode. More and more people start to use it in testnet. But you need to be careful with it before security audit report comes out.
Code: https://github.com/cosmos/ledger-cosmos
But we have to admit that storing Atom with Ledger is not very handy for validators, because you need to actually push a button to do a self-delegate. So, no automation of self-bond is not possible with Ledger. In this way, Ledger is more suitable for delegators than validators.
Validator Key
To understand the role of validators, you need to understand Tendermint consensus first. Tendermint is a Byzantine fault tolerance State Machine Replication (SMR) for arbitrary deterministic, finite state machines. It could be used to store distributed ledger of blockchain in individual node.
Validators are responsible for committing new blocks in the blockchain. These validators participate in the consensus protocol by broadcasting votes which contain cryptographic signatures signed by each validator’s private key. These private keys are associated to amount of voting power. Validators are expected to be online, as a block will be committed when +2/3 of the total voting power signed precommit votes for that block at the same round. The +2/3 set of precommit votes is called a polka.
Sounds complicated? Here is a visual for consensus: https://github.com/kevlubkcm/tenderflow Check this out http://3.80.184.123 and I am sure you will have a better image.In Game-of-Stakes, there are more than 200 validators. http://nylira.net provides you with a cool visualization.
Proof-of-Stake mechanism is implemented in Cosmos Network. Validators’s voting power equals the amount of tokens they staked in system. They bond their currency holdings in a security deposit that can be inflated to a certain percentage, but also be destroyed/slashed if there are evidence about malicious behavior in the consensus protocol. Unlike PoW systems, attacker needs to invest a lot hash powers to fork the chain. In PoS system, if attackers controls the private key of validators who hold +1/3 voting power, they could attack/fork the chain easily. Now you may understand that validator’s private key is the most important asset.
In Cosmos network, there are two main slashing condition: downtime & double-signature.
Downtime slashing
As discussed above, a block can only be committed when +2/3 of the validator agree on a new block. So, all validators are required to stay online. In Slashing module of Cosmos Hub, there is a condition for punishing validators for not signing enough blocks within a certain period.
To mitigate the risk of downtime slashing, some validators look into the solution of highly available, HA architecture. Under HA solution, there exist several gaiad instances which using the same private key. Then, the votes are broadcasted to network under control of a crash tolerant model (Raft, or Paxos based). More discussion could be found here.
However, the failure of a crash tolerant model will lead to double signature from the same private key, which is seen as a significant malicious behavior.
Double signature Slashing
Double signatures lead to conflicting blocks and votes, which may lead to fork of chain or double-spend. Key Management Service, or KMS, is a service which attempts to prevent validator from double-signing by making the KMS aware of the current block height. Hardware Security Module, or HSM is an implementation of KMS, which can store a private key in some hardware and ensure in a secure setting that it won't sign the same block twice.
When we take a close of the Tendermint consensus, the signing part is delegated to a process that communicates with Tendermint over SocketPV network interface.
The signing process is deterministic state machine whose responsibility is signing protocol message and ensuring that the validator does not double sign. HSM must support ed25519 signatures for Cosmos hub. The most popular HSM in the market is YubiHSM2. It supports ed25519and can protect a private key but cannot ensure in a secure setting that it won't sign the same block twice.
Two KMS solution
There are already two open-sourced KMS available. Tendermint team is also working on extending our Ledger Nano S application to support validator signing. All KMS solutions can store recent blocks and mitigate double signing attacks.
- Tendermint KMS
Code: https://github.com/tendermint/kms
Tendermint KMS, or tmkms is an implementation of PrivateValidator interface which handles signing and implements double signing protection of messages:Proposal, PreVotes, PreCommits. It is fully written in Rust.
tmkms could be used for a range of signers: YubiHSM2, Ledger Nano S and others. It is very extensible and flexible.
Feature:
- High-availability access to validator signing keys
- Double-signing prevention even in the event the validator process is compromised
- Hardware security module storage for validator keys which can survive host compromise
- Ability to sign several chains using a single highly secured light weight service
Tutorial is here: https://forum.cosmos.network/t/ann-tendermint-kms-v0-2-validator-signing-support/1361/21
How to install: https://github.com/tendermint/kms#installation
Many validators have already tested it on Game of Stakes, and the feedbacks are pretty good.
Aiakos by Certus One
Certus one is a team of dedicated blockchain , systems engineering and security experts focused on blockchain app development and infrastructure. They see tmkms introduce man-in-the-middle attacks in their knowledge base. Their KMS solution: Aiakos uses the PrivValidator interface of Tendermint to implement a direct wrapper to the YubiHSM2.
In order to use Aiakos, you have to install the Yubico YubiHSM2 connector on your host and apply a small patch your Cosmos node source to register it. It is a good practive that a validator should always build their own cosmos-sdk binaries
Code：https://github.com/certusone/aiakos
How to use:https://github.com/certusone/aiakos#usage
Aiakosi doesn’t implement double signing protection in the event of a validator compromise. While with tmkms, HSM could be separated from validator infrastructure. But the community needs to find more HSM solutions.
Note: all these solutions are not production ready, you could test any of them on testnet first.
Future Improvements
YubiHSM2 is the most commonly used HSM among Cosmos validators. It signs messages with private key with Ed22519. But it is limited to the size of one message to the YubiHSM: 2019 bytes. This limitation could break something when the consensus/transaction message is too long.
Reference
About the author
I am a blockchain engineer working for irisnet project. Currently, I am the testnet coordinator for IRIS and is a Cosmos validator. I regularly write down my thoughts. I am also interested in cryptoeconomics and mechanism design.
