EOSphere Blog
Published in

EOSphere Blog

WAX Technical How To #7

In a previous Technical How To article the process of building a reliable and solid WAX Mainnet API and P2P node was walked through.

There is however another nodeos implementation type that is crucial for the operation of numerous WAX Mainnet Services such as Hyperion Full History and the Atomic Assets API and that is the State-History service.

This 7th WAX Technical How To Edition will show you how to build, configure and deploy nodeos to provide State-History functionality also known as a State History Protocol (SHIP) node.

How to Set Up a WAX State-History Node

The State-History EOSIO plugin is used by nodeos to capture historical data about the blockchain state (WAX Mainnet in this case) and store this data in an externally readable flat file format.

This plugin opens a websocket interface that can be utilised by multiple simultaneous external services such as Hyperion and Atomic to simply and effectively read this data.

Considering how reliant external services are on SHIP nodes, it is crucial that a Guild provide a solid reliable service that can scale to meet the demands of the expanding requirements of the WAX Mainnet.

This article will walk through an example that is currently valid (January 2022) and in use by the EOSphere Guild who are serving multiple websocket connections for both our own services and services for other Guilds.

Mainnet Requirements

Hardware

  • 4 Core CPU / 4Ghz+ recommended if you would like to produce blocks
  • (1) 256GB+ Disk / Enterprise Grade SSD or NVMe (High Endurance Req.)
  • (2) 4TB+ Disk / SAS or SATA are OK however SSD or NVMe preferred

Currently (January 2022) the blocksand state-history directories take up just short of 4TB of disk space.

980G    ./blocks
3.0T ./state-history
  • 64GB+ RAM

Operating System

  • Amazon Linux 2
  • CentOS 7
  • CentOS 7.x
  • CentOS 8
  • Ubuntu 16.04
  • Ubuntu 18.04 (Recommended)
  • Ubuntu 20.04 (Recommended)
  • MacOS 10.14 (Mojave)
  • MacOS 10.15 (Catalina)

Internet

  • Modern Broadband / Fibre Connection (100Mb/s synchronous and above)

Prepare the Operating System Environment

Before the WAX software is built and configured, the operating system environment Ubuntu 18.04 in this case needs to be configured for performance and the load that is will take on.

Zettabyte File System (ZFS)

This node build uses 2 Discrete SAS Disks in order to balance disk IO and provide a more affordable storage option for the /blocks and /state-history directories which together are currently 4TB and growing.

Disk 1 is the high speed enterprise grade SSD or NVMe will be the OS disk used for the WAX software, all config and the state files. The state files are extremely IO intensive, a consumer based SSD’s life span will be short lived due to the high number of writes hence the need to use a high endurance enterprise SSD or NVMe.

Note: It needs to be mentioned that there are mechanisms to run these state files in memory if you have enough available, this topic will be covered in a future article.

In this example Disk 1 will run the default OS Ext4 File System that will already be implemented during the Ubuntu 18.04 installation.

Disk 2 is the large capacity SATA or SAS disk that will host the /blocks and /state-history directories. The IO demands on these directories are far lower than the state files and slower larger capacity spindle based disks are still suitable.

In this example Disk 2 will run the ZFS File System which will give us two main benefits. ZFS will enable us to use LZ4 compression and it will improve disk IO with Adaptive Replacement Cache (ARC). Compression will only be used for the /blocks directory (which currently is a gain of 1.3x ) as there on no benefits on the already optimised state-history flat file.

Implement ZFS on Disk 2 with the below configuration:

#Install ZFS
> sudo apt-get install zfsutils-linux
#Locate the Disk 2 device name
> lsblk
#Create ZFS Pool called "datavolume" on device "sdb"
> sudo zpool create datavolume /dev/sdb
#Enable LZ4 compression
> sudo zfs set compression=lz4 datavolume
#Disable ZFS access time Updates
> sudo zfs set atime=off datavolume
#Set ARC to only cache metadata
> sudo zfs set primarycache=all datavolume
#Set the mountpoint location to your preferred location
> sudo zfs set mountpoint=/home/eosphere/datavolume datavolume
#Create the specific folders for /blocks and /state-history
> mkdir /home/eosphere/datavolume/blocks
> mkdir /home/eosphere/datavolume/state-history#Set specific mountpoint for /blocks and /state-history
> sudo zfs create -o mountpoint=/home/eosphere/datavolume/blocks datavolume/blocks
> sudo zfs create -o mountpoint=/home/eosphere/datavolume/state-history datavolume/state-history#Turn off lz4 compression on the state-history mountpoint
> sudo zfs set compression=none datavolume/state-history
#Verify ZFS Settings
> zfs get all

Network Time Protocol (NTP)

It is crucial for a globally meshed blockchain to have synchronised time across all nodes.

Chrony is an excellent NTP client and is quite suitable for the needs of WAX Mainnet.

Install, configure and verify as below:

#Install Chrony
> sudo apt install chrony
#If necessary manually add local peers, these are AU servers
> sudo nano /etc/chrony/chrony.conf

server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
server 3.pool.ntp.org
#Restart Chrony
> sudo /etc/init.d/chrony restart
#Verify
> chronyc sources -v
> chronyc tracking
#Set your local timezone if necessary
> sudo timedatectl set-timezone Australia/Perth

Stack Limits and Openfiles

WAX software memory addressing and the number of API requests a Production Mainnet Node will receive require that the Ubuntu 18.04 default Stack Limit and Number of Open Files be increased.

Configure and verify the raised limits as below:

> sudo nano /etc/systemd/system.conf#Append the following configuration
DefaultLimitNOFILE=64000
DefaultLimitSTACK=65536000
#Restart server and verify
> ulimit -a

Build the Software

The WAX software is derived from opensource EOSIO software, however it has been modified to suit the needs of the WAX Protocol Network.

Currently the WAX Block Producer accepted software build and version is v2.0.13wax01 created by cc32d9 who is member of the EOS Amsterdam Guild

The latest waxbuild tag is always available on the WAX Github however there may be issues with reliable operation of v2.1.x software so stick with v2.0.x for now (January 2022)

Building Process

This example uses Ubuntu Linux to build the WAX software from source following the process below:

> cd ~> git clone https://github.com/worldwide-asset-exchange/wax-blockchain.git> cd wax-blockchain> git checkout v2.0.13wax01> git submodule update --init --recursive> cd scripts > ./eosio_build.sh -P./eosio_install.sh #If you want to install. Binaries are in ~/wax2.0/build/programs

As the -P option is used on the build script this may take some time to compile as it will build with pinned clang and libcxx

Configuration

Now that a clean build of the WAX software has been compiled lets get into configuring state-history for WAX Mainnet operation.

To configure and start the node nodeos will be used, nodeosis the core service daemon that runs on every WAX Protocol Network node.

nodeos can be configured to process smart contracts, validate transactions, produce blocks containing valid transactions, and confirm blocks to record them on the blockchain. You can read more about it here.

The primary operational functions of nodeos are; running it as a Block Producer, Network API Endpoint, P2P Seed Node or State-History Node. Typically on a busy network such as the WAX Mainnet you would separate these functions across physically discrete servers.

In this WAX Mainnet example you will make your node connect to other network peers and available for API and Websocket connections as a State-History (SHIP) node.

nodeos requires two files to connect to peers and run on the WAX Mainnet:

config.ini

Create a default config.ini by running nodeos without config as per the command below:

> mkdir ~/waxdata> nodeos --data-dir ~/waxdata --config-dir ~/waxdata

You will then be able to edit the newly created config.ini and see all the available parameters:

> cd ~/waxdata> nano config.ini

Now edit the config.iniand add the following configuration settings:

# the location of the blocks directory on Disk 2
blocks-dir = /home/eosphere/datavolume/blocks
wasm-runtime = eos-vm-jit
chain-state-db-size-mb = 131072
chain-state-db-guard-size-mb = 1024
read-mode = headhttp-server-address = 0.0.0.0:8888
access-control-allow-origin = *
access-control-allow-headers = Origin, X-Requested-With, Content-Type, Accept
http-max-response-time-ms = 100
verbose-http-errors = true
http-validate-host = false
p2p-listen-endpoint = 0.0.0.0:9876# 3dkrenderwax: FI, wax-peer
p2p-peer-address = peer.3dkrender.com:9880

# 3dkrenderwax: FI, query
p2p-peer-address = query.3dkrender.com:9880

# alohaeosprod: US, Oregon
p2p-peer-address = peer.wax.alohaeos.com:9876

# amsterdamwax: NL, Amsterdam
p2p-peer-address = wax.eu.eosamsterdam.net:9101

# blacklusionx: DE, Germany
p2p-peer-address = peer1.wax.blacklusion.io:4646

# blokcrafters: US, Portland, Oregon
p2p-peer-address = wax-peer-us.blokcrafters.io:9876

# blokcrafters: FI, Helsinki, Uusimaa
p2p-peer-address = wax-peer-eu.blokcrafters.io:9876

# bp.wecan: DE, London
p2p-peer-address = seed2-wax-mainnet.wecan.dev:14998

# cryptolions1: DE, Germany-Finland
p2p-peer-address = wax.cryptolions.io:9876

# dapplica: DE, Germany-Finland
p2p-peer-address = wax.dapplica.io:9876

# eosarabianet: DE, Munich
p2p-peer-address = p2p-wax.eosarabia.net:9876

# eosauthority: DE, Falkenstein
p2p-peer-address = node-wax.eosauthority.com:10301

# eosauthority: FI, Helsinki
p2p-peer-address = node-wax-p2p.eosauthority.com:10301

# eosdacserver: GB, United Kingdom
p2p-peer-address = wax-p2p.eosdac.io:29876

# eosdublinwow: FI, Finland
p2p-peer-address = wax.p2p.eosdublin.io:9876

# eoseouldotio: JP, Seoul
p2p-peer-address = p2p.wax.eoseoul.io:29876
# eosphereiobp: AU, Sydney
p2p-peer-address = peer1-wax.eosphere.io:9876
# eosphereiobp: AU, Sydney
p2p-peer-address = peer2-wax.eosphere.io:9876

# greeneosiobp: DE, Germany
p2p-peer-address = p2p1.wax.greeneosio.com:9876

# ledgerwiseio: FI, LB
p2p-peer-address = waxp2p.ledgerwise.io:21877

# nation.wax: CA, Canada
p2p-peer-address = wax.eosn.io:9876

# niftylifewax: FI, Helsinki, Finland
p2p-peer-address = p2p2.niftylife.io:9876

# sentnlagents: GB, United Kingdom
p2p-peer-address = waxp2p.sentnl.io:9876

# tokengamerio: DE, Germany
p2p-peer-address = peer.tokengamer.io:9876

# wax.eastern: DE, Frankfurt
p2p-peer-address = p2p.waxeastern.cn:9876

# waxmadrid111: DE, SEED
p2p-peer-address = wax-seed.eosiomadrid.io:9876

# waxswedenorg: SE, Sweden
p2p-peer-address = p2p.waxsweden.org:35777
# PeerList - https://validate.eosnation.io/wax/reports/config.htmlagent-name = "<yourname> WAX Mainnet State-History"

sync-fetch-span = 500
state-history-dir = /home/eosphere/datavolume/state-history
trace-history = true
chain-state-history = true
state-history-endpoint = 0.0.0.0:8080
plugin = eosio::http_plugin
plugin = eosio::state_history_plugin
plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin

genesis.json

These are the initial state parameters required by every new starting node on the WAX Mainnet. Create the file as below:

> cd ~/waxdata> nano genesis.json

Add the following parameters to the genesis.json file for the WAX Public Mainnet:

{
"initial_timestamp": "2019-06-05T12:00:00.000",
"initial_key": "EOS8i2pkwtv2JmdYWNJdcy5BcJ7wCE5q6mpE1hwT25HdgHMzeRday",
"initial_configuration": {
"max_block_net_usage": 1048576,
"target_block_net_usage_pct": 1000,
"max_transaction_net_usage": 524288,
"base_per_transaction_net_usage": 12,
"net_usage_leeway": 500,
"context_free_discount_net_usage_num": 20,
"context_free_discount_net_usage_den": 100,
"max_block_cpu_usage": 500000,
"target_block_cpu_usage_pct": 2000,
"max_transaction_cpu_usage": 150000,
"min_transaction_cpu_usage": 100,
"max_transaction_lifetime": 3600,
"deferred_trx_expiration_window": 600,
"max_transaction_delay": 3888000,
"max_inline_action_size": 4096,
"max_inline_action_depth": 4,
"max_authority_depth": 6
}
}

Running Nodeos

Now that the config.ini has been configured and the initial WAX Mainnet chain parameters genesis.json have been created, you can now join the network and sync up the node.

Use screen to keep your session live even when you disconnect, usage below:

#Create a new screen session> screen -US wax #Disconnect screen session> ctrl-a+d #Reconnect screen session> screen -r wax

Run nodeos with pointers to the config, data directory and genesis file:

nodeos --data-dir ~/waxdata --config-dir ~/waxdata --genesis-json=~/waxdata/genesis.json --disable-replay-opts

Your WAX Mainnet node will now start syncing with the configured peers and catch-up with the WAX Mainnet chain headblock.

At time of writing (January 2022) this may take up to 2 weeks for a complete block sync from genesis. It may also help if you choose a few closely located peers to limit peer overload and ensure low latency.

As your node syncs from the start of the chain it will build the log and index files in the /blocks and /state-history directories.

Your node will be accessible via http port 8888 for queries and ws port 8080 for state-history.

Be sure to ask any questions in the EOSphere Telegram

EOSphere Guild is a Block Producer on the WAX Protocol Network as well as many other EOSIO based Blockchains.

If you find our work helpful, please vote for us on the WAX Mainnet: eosphereiobp

If you prefer to proxy your vote, our proxy account is : blklotusprxy

Connect with EOSphere via these channels:

TELEGRAM | MEDIUM |YOUTUBE | FACEBOOK | TWITTER | INSTAGRAM

--

--

--

Leading Australian EOSIO Block Producer candidate serving the entire EOSphere

Recommended from Medium

Sensibly Sized Services — part 1

The Rise of Roam Research

RTC Conversation with Lakshmi Subbramanian: Becoming an Engineering Manager at Goldman Sachs

It was, in the words of Dr. Daniel Jacobson lead researcher and chief scientist

Rails Rspec request spec session handling

A Post-Bootcamp Love Letter

Thoughts on Refactoring

Use Nvidia-Docker to never worry about installing a specific version of a framework again

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ross Dold

Ross Dold

Co-Founder of EOSphere Australia

More from Medium

WAX Technical How To #8

bountyblok on WAX — NOVEMBER 2021 UPDATE

MetaSoccer x Cryptocity: Widening the Net

How To Earn WAX Rewards By Voting and Staking Your Extra WAX