Behind the Scenes at HOPR Tech Day
The entire HOPR team met in Zurich in the last week of July. As a decentralized project with team members spread across the globe, it’s a rare event to have everyone together at once, so we wanted to make the most of it.
We had a lot of amazing discussions and events, but today I want to share a behind-the-scenes look at our tech team architecture discussion. The results will have far-reaching consequences for how HOPR nodes work and help us to build more stable releases faster.
We met at Zurich’s Technopark for a full day of discussion, brainstorming and design. The result was some major overhauls in the architecture of the protocol and how we code, test and manage releases.
Refactoring core-ethereum
Our most important decision was to refactor the core-ethereum component within the HOPR protocol. As the name suggests, core-ethereum acts as a bridge between the core (libp2p, connectivity) and ethereum (smart contracts) components. It’s responsible for lots of crucial parts of the HOPR protocol, including assessing the current state of the payment channel topology, broadcasting and indexing transactions to and from the blockchain, and managing the locally stored database.
It used to look like this:
This was a pleasingly simple, streamlined design, but unfortunately the simplicity obscured some issues: it could be hard to tease out the sub-components to work on in a granular fashion, and some sub-components that really belonged in core-ethereum were stranded in other parts of the protocol. We wanted a new design that really specified all the different parts and how they interact, allowing the team to understand the whole more clearly and work more easily on their individual areas of focus.
The figure below shows the new design in all its glory!
Although it may look more complicated, separating out these components and giving them all a clearly defined purpose will make development and testing much, much easier.
One crucial factor in this architecture design is that it’s compatible with our long-term goal to migrate to Rust. Unlike the previous design, this setup can be smoothly migrated to Rust while maintaining almost identical interfaces between components.
This was the major outcome of the tech team architecture discussion, but we also made some other important decisions which will improve our development processes and help us to produce better, more stable releases more quickly.
Improvements to Release Processes, Testing, and Coding Practices
We’ll be updating the codebase to include various new tooling, including improved code coverage, linting and lots more automation.
HOPR releases are a major event with a LOT of steps and testing. As always with complex development, sometimes things go wrong and you have to go back to earlier parts of the process.
We already have heavy automation here, but with the old setup there were still some manual steps here which could cause bottlenecks, especially with team members in so many different timezones. We’ll be automating these so we can save the tech team a lot of time in the long run. In cases where things don’t go to plan in the pipeline, PRs will automatically be created so they can be more easily addressed.
Good news for our valued AVADO community is that we’ll be implementing automatic node migration. When we push new public versions to AVADO, all nodes will automatically update to the new version. Of course, before we implement this we’ll be running stringent tests to ensure there are no compatibility issues and user funds are safe at all times.
Finally, we’ve changed our protocol identifiers on the protocol level so different releases in the same environment can talk to each other.
When we first began developing HOPR, nodes from different versions couldn’t talk to each other, even if the difference between those versions was minor. Regular participants in our testnets will probably remember the dance of reinstalling their node when hotfixes were pushed.
We later introduced environments (named after cities), but this behaviour still persisted, even though it didn’t need to. Obviously this doesn’t make for a great user experience, so we’ll remove this behaviour so we can re-use environments when we make small updates to releases. Major updates will require a new environment, and nodes still won’t be able to communicate across environments, but this will be much less frequent.
This was a great day with the tech team firing on all cylinders to bring improvements that will pay off handsomely in the short and long term.
This work all happens deep under the hood, so users and community members are unlikely to directly see any of the results of this refactoring, but everyone will benefit from reduced release times, more stable nodes and the ability to support a bigger network for longer!
As always, please feel free to check out our progress on Github, or via our bi-weekly tech updates on Twitter.
Steven Noni,
HOPR Software Engineer and Bounty Master
Website: https://www.hoprnet.org
Twitter: https://twitter.com/hoprnet
Telegram: https://t.me/hoprnet
Discord: https://discord.gg/dEAWC4G
Forum: https://forum.hoprnet.org
Staking: https://stake.hoprnet.org
Bounties: https://bounties.hoprnet.org
