Open in app

Sign in

Write

Sign in

Setting Up a Agoric Validator Node Using TMKMS

Calin Tatar
6 min readJun 29, 2023

Agoric is a platform for smart contracts that uses the Tendermint Proof-of-Stake (PoS) mechanism and a native asset to secure the network. It is built on the Cosmos SDK and allows developers to create decentralized applications using JavaScript smart contracts. In addition, Agoric provides built-in building blocks for decentralized finance (DeFi), which enables it to support a second local currency for facilitating economic activity and maintaining a healthy ecosystem. Agoric chain offers a comprehensive solution for creating and operating decentralized applications with a focus on supporting DeFi.

Hardware requirments

Chain validators should operate a node with the following recommendations in mind:

  • An all-purpose server with at least 16 GB RAM
  • Supported processor architecture with at least 4 cores/ vCPU
  • x86_64 CPUs are recommended as there known issues with arm64 compatibility. We aim to fix these issues in the long term, but are currently focused on network launch.
  • High-performance SSD, e.g. NVMe SSD with storage that accommodates the following estimated value with a block cadence of 5 seconds/block:
  • a full node (with the default pruning=syncable) grows at the rate of ~6GB_month (approximately 200MB_day)
  • an archiving node (with pruning=nothing) grows at the rate of ~12GB_month (approximately 400MB_day)

Installation

Setup validator name

Replace YOUR_MONIKER_GOES_HERE with your validator name

MONIKER="YOUR_MONIKER_GOES_HERE"

Install dependencies

ADD PACKAGE REPOSITORY FOR NODE.JS

curl -Ls https://deb.nodesource.com/setup_16.x | sudo bash

ADD PACKAGE REPOSITORY FOR YARN

curl -Ls https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

UPDATE SYSTEM AND INSTALL BUILD TOOLS

sudo apt -q update
sudo apt -qy install curl git jq lz4 build-essential nodejs=16.* yarn
sudo apt -qy upgrade

INSTALL GO

sudo rm -rf /usr/local/go
curl -Ls https://go.dev/dl/go1.20.5.linux-amd64.tar.gz | sudo tar -xzf - -C /usr/local
eval $(echo 'export PATH=$PATH:/usr/local/go/bin' | sudo tee /etc/profile.d/golang.sh)
eval $(echo 'export PATH=$PATH:$HOME/go/bin' | tee -a $HOME/.profile

Download and build binaries

# Clone project repository
cd $HOME
rm -rf mainnet1B-rc3
git clone https://github.com/Agoric/agoric-sdk.git mainnet1B-rc3
cd mainnet1B-rc3
git checkout mainnet1B-rc3
# Install and build Agoric Javascript packages
yarn install && yarn build# Install and build Agoric Cosmos SDK support
(cd packages/cosmic-swingset && make)

Initialize the node

# Set node configuration
agd config chain-id agoric-3
agd config keyring-backend file
# Initialize the node
agd init $MONIKER --chain-id agoric-3# Download genesis and addrbook
curl -Ls https://snapshots.kjnodes.com/agoric/genesis.json > $HOME/.agoric/config/genesis.json
curl -Ls https://snapshots.kjnodes.com/agoric/addrbook.json > $HOME/.agoric/config/addrbook.json# Add seeds
sed -i -e "s|^seeds *=.*|seeds = \"400f3d9e30b69e78a7fb891f60d76fa3c73f0ecc@agoric.rpc.kjnodes.com:12759\"|" $HOME/.agoric/config/config.toml# Set minimum gas price
sed -i -e "s|^minimum-gas-prices *=.*|minimum-gas-prices = \"0.025ubld\"|" $HOME/.agoric/config/app.toml# Set pruning
sed -i \
  -e 's|^pruning *=.*|pruning = "custom"|' \
  -e 's|^pruning-keep-recent *=.*|pruning-keep-recent = "100"|' \
  -e 's|^pruning-keep-every *=.*|pruning-keep-every = "0"|' \
  -e 's|^pruning-interval *=.*|pruning-interval = "19"|' \
  $HOME/.agoric/config/app.toml

Download latest chain snapshot

curl -L https://snapshots.kjnodes.com/agoric/snapshot_latest.tar.lz4 | tar -Ilz4 -xf - -C $HOME/.agoric
[[ -f $HOME/.agoric/data/upgrade-info.json ]] && cp $HOME/.agoric/data/upgrade-info.json $HOME/.agoric/cosmovisor/genesis/upgrade-info.json

Start service and check the logs

sudo systemctl start agd && sudo journalctl -u agd -f --no-hostname -o cat

Introducing the Tendermint Key Management System

Tendermint Key Management System (better known as TMKMS) is designed to make private validator keys more secure and prevent doubling signing when a backup node is already in place.

The backup node must be synced upfront and ‘priv_validator_laddr’ should already be specified.

TMKMS provides other signifigant advantages such as:

  • Improved security and risk management policies
  • Unified API and support for various HSM (hardware security modules)
  • Double signing protection (software or hardware based). The TMKMS should be on a separate system. For the local TMKMS you must install it and secure the ‘priv_validator_key’. Keep in mind that the ‘priv_validator_key’ should be saved in different places and deleted from the node both ways. It sould also be deleted if present on another node.

What is Double Signing?

Double signing happens when a validator submits two signed messages for the same block. This can happen if a node operator or infrastructure provider optimizes their node configuration to prevent downtime by having a highly available backup entity running at the same time as a primary entity. Double signing is looked at as an attack on the network.

Having TMKMS setup on a separate server that holds the private keys for a validator prevents this issue. Having the private keys held on a separate server helps to keep them protected and can route the signing to the signing validator. If something happens to that signing validator, all that needs to be done is point the TMKMS server to the backup node. This should be said to that the backup server needs to be a different machine, and for high security, in a different region all together or country all together.

For more details read here.

Software Prerequisites

You will need the following prerequisites:

  • ✅ Rust (stable; 1.56+): https://rustup.rs/
  • ✅ C compiler: e.g. gcc, clang
  • ✅ pkg-config
  • ✅ libusb (1.0+). Install instructions for common platforms
#OS update
sudo apt update
#Install rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shsource $HOME/.cargo/envsudo apt install git build-essential ufw curl jq snapd --yessudo apt install libusb-1.0-0-devexport RUSTFLAGS=-Ctarget-feature=+aes,+ssse3

1. Download and Create TMKMS Binary

We will compile the software from the source code on GitHub and utilizing the ‘features=softsign’ flag. You also have the option to use the ‘features-yubihsm’ instead, which enables the use of a YubiKey.

Please note that Ledger support is currently malfunctioning at the time of writing this guide and we will no provide instructions for using YubiKey.

cd $HOME
git clone https://github.com/iqlusioninc/tmkms.git
cd $HOME/tmkms
cargo install tmkms --features=softsign
tmkms init config
tmkms softsign keygen ./tmkms/config/secrets/secret_connection_key

2. Copy the ‘priv_validator_key’ to the TMKMS Server, Then Import.

scp user@"nodeip":~/.agoric/config/priv_validator_key.json ~/tmkms/config/secrets
#if no connection possible, the key can also be copied over by winscp(windows) or cyberduck(mac)tmkms softsign import $HOME/tmkms/config/secrets/priv_validator_key.json $HOME/tmkms/config/secrets/priv_validator_key

3. Modify Config for TMKMS

Verify the Tendermint version with ‘binary tendermint version’.

Alternative configuration examples can be read here.

nano $HOME/tmkms/config/tmkms.toml
# Tendermint KMS configuration file## Chain Configuration### Agoric Network[[chain]]
id = "agoric-3"
key_format = { type = "bech32", account_key_prefix = "bldpub", consensus_key_prefix = "agoricvalconspub" }
state_file = "/root/tmkms/config/state/priv_validator_state.json"## Signing Provider Configuration### Software-based Signer Configuration[[providers.softsign]]
chain_ids = ["agoric-3"]
key_type = "consensus"
path = "/root/tmkms/config/secrets/priv_validator_key"## Validator Configuration[[validator]]
chain_id = "agoric-3"
addr = "tcp://"ipaddresnode":26680" # your validator node ip and port
secret_key = "/root/tmkms/config/secrets/secret_connection_key"
protocol_version = "v0.34"
reconnect = true

4. Create Service for TMKMS

sudo nano /etc/systemd/system/tmkms.service
[Unit]
Description=tmkms
After=network-online.target
[Service]
User=user
ExecStart=/home/user/.cargo/bin/tmkms start -c /home/user/config/tmkms.toml
Restart=on-failure
RestartSec=3
LimitNOFILE=4096
[Install]
WantedBy=multi-user.target

5. Enable the Service

sudo systemctl enable tmkms.service

6. Start the Service

sudo systemctl start tmkms

7. Check the Logs

sudo journalctl -u tmkms -f

8. Login To Your Validator and Open the ‘config’

#You need to enable KMS access by editing .agoric/config/config.toml. In this file, modify priv_validator_laddr to create a listening address/port or a unix socket in agoric
nano $HOME/.agoric/config/config.toml
#change the following lines
priv_validator_laddr = "tcp://0.0.0.0:26680"
# priv_validator_key_file = "config/priv_validator_key.json"# priv_validator_state_file = "data/priv_validator_state.json"

9. Allow Port 26680 in the Validator For TMKMS IP

sudo ufw allow from "ip address tmkms server" to any port 26680 proto tcp

10. Stop the Validator and Start TMKMS

sudo systemctl start tmkms.service && sudo journalctl -fu tmkms.service

Verify that the following lines are shown:

2022-03-08T23:42:38.931428Z ERROR tmkms::client: ["chainid"@tcp://ipaddresnode:26680] I/O error: Connection refused (os error 111)
2022-03-08T23:42:39.931729Z  INFO tmkms::connection::tcp: KMS node ID: 948f8fee83f7715f99b8b8a53d746ef00e7b0d9e
2022-03-08T23:42:39.932417Z ERROR tmkms::client: ["chainid"@tcp://ipaddresnode:26680] I/O error: Connection refused (os error 111)
2022-03-08T23:42:40.932732Z  INFO tmkms::connection::tcp: KMS node ID: 948f8fee83f7715f99b8b8a53d746ef00e7b0d9e
2022-03-08T23:42:40.933425Z ERROR tmkms::client: ["chainid"@tcp://ipaddresnode:26680] I/O error: Connection refused (os error 111)

11. Start The Validator, Check Logs on the TMKMS

Verify that the logs look similar to the example here:

2022-03-08T23:46:06.208451Z  INFO tmkms::connection::tcp: KMS node ID: 
2022-03-08T23:46:06.210568Z  INFO tmkms::session: [chainid@tcp://ipaddresnode:26680] connected to validator successfully
2022-03-08T23:46:06.210604Z  WARN tmkms::session: [chainid@tcp://ipaddresnode:26680]: unverified validator peer ID! ()
2022-03-08T23:46:15.929787Z  INFO tmkms::session: [chainid@tcp://ipaddresnode:26680] signed PreCommit:<nil> at h/r/s 3399910/0/2 (0 ms)

--

--

Calin Tatar

Written by Calin Tatar

1 Follower

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams