HOPR Säntis Update: Over 400 Nodes in 4 Days
Our HOPR Säntis incentivized testnet launched less than a week ago, and the response from the community has been amazing, with hundreds of people responding to our call to help contribute to online privacy and earn HOPR tokens. At the time of writing, 350 people have verified their node with the bot that oversees the testnet and received their first points on the scoreboard, and there are 100–200 nodes online day and night. We’ve had over 6,000 calls to our bot as it opens payment channels and sends out tokens for relayed data. Our Telegram and Discord channels have exploded, with people posting around the clock.
This post is just a brief update to report on the incredible engagement we’ve seen some far, some explanations and troubleshooting for the three most common issues users are facing, and a rundown of what we have planned for the rest of the testnet, which is due to finish on October 6th.
Status Update
A huge thank you to everyone who has participated so far. This is bleeding-edge tech and we know it’s not always the most stable or easy to use. We really appreciate the time everyone has taken to learn how to download and run a node. The data we’re gathering is making it much easier for us to spot problems in our network stack and help us
We’re also delighted that so many people are running nodes on their home devices, with around a 50–50 split between people running their node at home and in the cloud. We’re determined for HOPR to break out of the crypto pattern of having “decentralized” networks which are actually all reliant on the same handful of cloud providers, so this is amazing to see.
Since this is our first major testnet, and the first time the HOPR protocol has been so strenuously stress-tested, it’s unsurprising that there have been some hiccups. Thank you to everyone who has reported problems in our Telegram and Discord channels and who took the time to fill out our feedback form.
A lot of people have managed to verify their node and relay data without trouble, but there are three main issues affecting the rest of you. I’d likely to quickly go through these, explain what the problem is, whether there’s anything you can do to try and resolve it right now, and explain the steps we’re taking to fix them and how long that’s likely to take.
Problem 1 — Timeout
The most common problem seems to be an inability to establish a connection with the bot.
Troubleshooting: First, try to determine whether you just can’t reach the bot, or if you can’t reach any nodes at all. Type:
ping 16Uiu2HAmRE4fVtp8dF6H62NzRcx6LGUTL5fBRTdnAfZXjveP5Kz9And see if you get a pong back. Then try to ping another node. Ask in Discord for someone’s address, or copy one of the addresses from the scoreboard. If it’s just the bot that you can’t connect, then it’s possible it’s just down. If you wait a while it should restart.
If you can’t reach any nodes, or the problem reaching the bot persists for hours, try restarting your node. If that doesn’t help, some users have reported that using the crawl command until the number of connected peers is close to the number of connected nodes (visible at saentis.hoprnet.org) can help resolve this. The more of the network your node knows about the easier it is to communicate across it.
What we’re doing to fix this:
Most people seem to be able to resolve this issue with some effort, but we appreciate it’s annoying. We’re working on general improvements to network stability which should reduce the frequency of these timeouts, but these will take a little while to implement. The HOPR network stack is designed to be private, resilient and completely decentralized, and we’re not prepared to compromise on any of those for a quick fix.
In the meantime, we’re also planning to relax the timeout durations a little, to see if that will help. We’re looking at the data we’ve gathered about nodes experiencing timeout issues to try and determine the root cause.
Problem 2 — Score not increasing
The vast majority users are able to verify their node with the bot and receive the initial 100 points, but for many their score never increases after that.
Troubleshooting: The most benign explanation here is that you’ve just been unlucky. With over 400 registered nodes, and rewards only being sent out every 30s, it’s not that unlikely that you simply haven’t been randomly chosen. Just waiting should end this unlucky streak.
But if you’ve been online for a day and still not scored any points, then this likely indicates a problem. Sometimes, the bot can reach you but it can’t receive a message back. In this case, you’ll get a message from the bot explaining the problem. First, make sure you can ping the bot and other nodes. Then, ask someone in our Discord if they can ping your node, to check it’s reachable. If not, then restarting your node or using the crawl command may help.
What we’re doing to fix this: We’re looking into increasing the frequency with which the bot relays data, which will help us to identify who is being unlucky and who has a genuine problem.
We’re also rebuilding the bot and scoreboard, with a new version planned for release on Friday. A PingBot will work in tandem with CoverBot to ensure that only nodes which are online are eligible for relaying (currently, if the bot randomly selects a node which is down, then nothing happens and no points are awarded to anyone). UI improvements to the leaderboard will make it clear who last got points and when, so it will be clearer when there’s a problem.
Problem 3 — Node dropped off leaderboard
Some people have found they can verify their node successfully, but at some point, their node will vanish from the leaderboard. This is primarily a bug with our bot, but it seems to be triggered by underlying network issues.
Troubleshooting: Unfortunately, there isn’t much you can currently do here, except for registering a new node.
What we’re doing to fix this: We’re going to rebuild the scoreboard and bot, and then migrate the current scores across to a new database, so no-one’s progress will be lost. Sadly this is a major fix, so we don’t think we’ll be ready before Friday.
Next Steps
We’re rebuilding parts of the bot and leaderboard and will launch an improved version. Unfortunately, these issues have highlighted some quite intricate problems within our network stack, so they’re not trivial to fix. This is great news in the long run because it brings us closer to our goal of making the HOPR protocol stable and scalable. But in the short run, it does mean that the next few days are unlikely to see any improvements. This is common in these kinds of tests, and we’re grateful to the community for their patience and understanding.
Our plan is to release an update by the end of the week. In the meantime, we’re brainstorming some ways to earn points using the current setup that everyone will be able to participate in, even if they’re currently experiencing technical problems. Stay tuned in Telegram and Discord for more details.
Thank You!
While it hasn’t always been smooth sailing, overall we’re extremely pleased with the first few days of the Säntis testnet. This is the first time the HOPR protocol has been put under so much pressure, and it’s held up pretty well, which is testament to the hard work of our dev team.
Thanks also to everyone in the community for signing up and keeping your nodes online, particularly the almost 150 of you who took the time to fill in our feedback form. We’ll be sending out some points and xDAI based on that feedback once we’ve processed it all.
Finally, a huge thanks to all the community members who have been providing support in our various channels, freeing up the dev team to work on improvements to our bot and core protocol. You’ve been amazing, and it’s really heartening to see so many people being helpful and supportive.
The testnet will run until October 6th. If you haven’t registered, it’s certainly not too late. Our bot and leaderboard updates should be released by Friday, and we’ll be providing more ways to earn points soon. Follow our Telegram and Discord channels to stay up to date.
