Astar Peers Program — Troubleshooting Guide and FAQ

Sequaja
6 min readJul 14, 2023

--

1. Checking Uptime Percentage
You can check the uptime percentage of your Astar node on https://telemetry.polkadot.io/. Please note that this is for the Astar node uptime, not the system uptime.

2. Checking Lifetime Uptime
If the node or OS restarts, the uptime refreshes. However, if the network goes down and then comes back up, the uptime doesn’t refresh because the Astar service keeps running.

3. Finding Guides
You can find guides on how to set up and run your Astar node on the following links:

4. Transaction Queue Increase and Parachain Idle
If you notice that the transaction queue is increasing or the parachain is idle, it could be because the client is prioritizing the node that needs sync. You can try warp sync of the relay chain using --sync warp. This is efficient for a quick sync but can only be done on a fresh database.

5. Reward Conditions
You will receive your reward if you meet the following conditions:

  • You live in the targeted regions.
  • The node you’re running is an archive node.
  • The node has run for 1 month without significant interruption (at least 80% uptime is required).
  • The node is connected to Telemetry.
  • The node is running on a Raspberry Pi 4 (or other small ARM machines), with at least 4GB RAM recommended.

6. Reward Amount
In this redesigned phase, we propose a reward of $30 in $ASTR tokens per person each month, for ten individuals over a span of five months.

7. Errors and Repeated Start and Stop
If you encounter errors and the Astar collator keeps starting and stopping, you could try the following steps described here or you might need to rebuild the filesystem. Make sure to follow the correct steps when downloading the snapshot and starting the Astar service.

(Download the snapshot)
# systemctl stop astar.service
# rm -rf chains/astar/db
# mkdir -p chains/astar/db/full
# tar xf astar_2023-07-08.tar -C /var/lib/astar/chains/astar/db/full/
# chown -R astar.astar /var/lib/astar/chains/astar/db
# systemctl start astar.service

8. Using Archive Snapshot for Faster Sync
If you’re tired of waiting for the sync, you can use an archive snapshot from Stakecraft. This will delete your actual database, so make sure to follow the correct steps when using the snapshot. You can find the Astar snapshot URL on the Stakecraft website: https://snapshots.stakecraft.com/

Warning: It’s still possible to use a snapshot (see docs) but I won’t encourage this option as this can cause troubles in the long term, much better to have a db synced on the node directly.

9. Node disappears from telemetry
Restart the PI to fix this issue.

10. brcmfmac: brcmf_sdio_hdparse: seq 218: max tx seq number error
This message comes from the WiFi driver (Broadcom). While it doesn’t immediately stop communication, it was followed by a network disconnection after a certain interval. The exact conditions for this issue are not clear, but rebooting the RasPi seems to resolve it for a while.

11. IPv6: MLD: clamping QRV from 1 to 2!
This message didn’t seem to have a significant impact on operation, but to eliminate unnecessary processes and for troubleshooting, the following settings were applied to disable IPv6 and multicast support:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
sudo sysctl -w net.ipv6.mld_qrv=1

12. astar-collator[19678]: 2023–05–31 09:23:17 [Parachain] :x: Error while dialing /dns/telemetry.polkadot.io/tcp/443/x-parity-wss/%2Fsubmit%2F: Custom { kind: Other, error: Timeout }
This is an occasional error seen in astar-collator. It’s a communication error with Telemetry, but it doesn’t interfere with the node’s essential operations. Therefore, this error can be ignored during synchronization.

13. Swap file is consistently in use
The Raspberry Pi model seems to be resource-intensive. The 4GB memory is quickly consumed after starting synchronization, and the swap file is consistently used, usually between several hundred MB to 1.XGB. This is manageable because the swap file is created on the SSD, but it would likely have a significant impact if it were on an HDD.

Normally, using swap is considered a “loss” in a system, so this state is not typically acceptable. Just before a hang-up, the swap file is usually saturated (fully consumed).

14. Will updates cause the node to turn off in the future?
No this shouldn’t impact the node process, maybe just slow it down a bit during the update process.

15. How can I not wait for 3+ more weeks to get caught up with the blocks?
It takes a while for the blocks to catch up on the PI. That is why the only option you will have is to wait. It takes long to sync. It’s still possible to use a snapshot (see docs) but I won’t encourage this option as this can cause troubles in the long term, much better to have a db synced on the node directly

16. Guide to Fixing UAS Driver Issue with SSD Enclosure on Raspberry Pi

Problem: The UAS (USB Attached SCSI) driver is causing disconnection issues with the SSD enclosure on a Raspberry Pi.

Symptoms: The SSD enclosure disconnects intermittently when connected to the Raspberry Pi.

a) Identify the Device:

  • Run lsusb to list all USB devices connected to your Raspberry Pi. Look for your SSD enclosure in the list.
  • Note the ID of the SSD enclosure. It should be in the format ID xxxx:yyyy. For example, ID 0bda:9210.

b) Create a Quirk Rule:

  • Open or create a configuration file to define a quirk rule. Use the command:
sudo nano /etc/modprobe.d/ignore_uas.conf

Add the following line to the file, replacing xxxx:yyyy with your device's ID:

options usb-storage quirks=xxxx:yyyy:u
  • Save and exit the editor (in nano, press CTRL+X, then Y, and Enter).

c) Update the Initramfs:

  • Update the initial RAM filesystem with the command
sudo update-initramfs -u

d) Verify the Fix:

  • Reboot your Raspberry Pi.
  • After rebooting, check the system messages with sudo dmesg.
  • Look for lines indicating that the Raspberry Pi is now using usb-storage instead of UAS for the SSD enclosure. You should see a message like "UAS is ignored for this device, using usb-storage instead."

Result: After applying this fix, your SSD enclosure should no longer disconnect due to issues with the UAS driver on your Raspberry Pi.

17. Set your on-chain identity

To improve reliability of future payments, we would like to ask you to set your on-chain identity. The process is very simple and can be found here: https://docs.astar.network/docs/build/nodes/collator/spinup_collator#identity. Once your identity is set on-chain, please ask for a judgment, Astar team will provide it.

18. Problems after Replacing HDD with SSD and how to fix

After replacing the HDD with an SSD problems can occur and the node won’t run correctly. A possible cause can be that the attributes of the data directory were incorrect. To fix this run the following commands:

# systemctl stop astar.service
# chown -R astar.astar /var/lib/astar
# systemctl start astar.service

19. Set-up on-chain identity for payments

To enhance the reliability of forthcoming payments, we kindly request that you establish your on-chain identity. The procedure is straightforward and detailed instructions can be found here. Once you have set your on-chain identity, please request a judgment, and the Astar team will promptly provide it. Upon successful judgment by the team, you will notice a green indicator in the Polkadot JS UI. This marks the completion of the process.

20. Save disk space using pruning for the polkadot database

The Astar client embeeds an Astar node and a Polkadot node. If you need to save space on your disk, you can prune the Polkadot database to much higher value. If you already have a node running, here is the procedure. This will save you ~300 Gb space on the disk.

stop the node:

sudo systemctl stop astar.service

update the astar service in /etc/systemd/system/astar.service to this

sudo nano /etc/systemd/system/astar.service

[Unit]
Description=Astar Archive node

[Service]
User=astar
Group=astar

ExecStart=/usr/local/bin/astar-collator \
--state-pruning archive \
--blocks-pruning archive \
--name {NODE_NAME} \
--chain astar \
--base-path /var/lib/astar \
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' \
-- \
--state-pruning 100 \
--blocks-pruning 100 \
--sync warp

Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Reload systemd daemon to account update

sudo systemctl daemon-reload

Delete the Polkadot database

sudo rm -rf /var/lib/astar/polkadot/chains/polkadot/db

Restart the node

sudo systemctl start astar.service

21. Monitoring the node

Monitoring on the node itself is nice for a start, but if Pi fails, you’ve got no alert. So you’d need another Pi for that, and a monitoring setup like this one: https://medium.com/bld-nodes/monitoring-multiple-substrate-nodes-polkadot-kusama-parachains-2022-2eb85cdf2533.

22. Addressing RAM Issues in Node Operations

Operating a node with 4GB RAM may not be sufficient for stable performance. The system could start to use swap memory and make it less performant. Users have reported frequent crashes after a few hours of operation, particularly as the blockchain reaches higher block heights and transaction volumes.

Recommendation: For a sustainable and stable node operation, upgrading to a mini PC or device with at least 8GB of RAM is advisable. This upgrade will help manage higher memory requirements and prevent frequent crashes and the node should always be monitored if possible.

--

--