Archipel Polkadot validators are Live!

Ostapenco Vladimir
Lugus Labs
Published in
5 min readSep 4, 2020

TL; DR: After Kusama canary Network testing, new security features, supervision, and monitoring deployed, we are pleased to announce that Archipel Validators are now live on the Polkadot network and are ready to validate. For speedy boarding, discover Archipel validators’ details at the end of this blog post. For smoother boarding, let’s begin with the full story.

Introduction

Archipel’s goal is to assure High-Availability for blockchain services on decentralized infrastructure at home by federating several DAppNodes.

That’s what drives us from the beginning of the Archipel project.

If you want to read more about our global vision and motivations, we have an Atoll paper available at Archipel.id.

Recently we have released the second version of our project called Archipel 2.0. You can read more about the release content by following this link.

In this article, we want to present our deployment of a secure and highly available Polkadot validator. We have tested it for several months on the Kusama network, and we tuned our deployment to get the best results.

Our Validators

In this section, we will explain how we are using Archipel to have a secure and highly available Polkadot validator setup with DappNodes at home.

Archipel Validators and fail-over

Following our vision, all our Polkadot validators are deployed with DappNode software and hardware at home. At this moment, we have 9 Polkadot validator instances running. 3 of them are active validator nodes. If something goes wrong with active validator nodes, we have 6 passive nodes ready to take the relay. They are distributed over 5 different locations, 3 cities, 2 countries, 2 continents, and 3 Internet service providers.

External Sentry Nodes

To make our validator setup secure, we are using external sentry nodes. Our validators are never exposed externally, and all blocks are propagated via external sentry nodes.

Our sentry nodes are deployed in Europe (Germany, Netherlands, France) at three different cloud providers.

We are aware that Sentry nodes will be deprecated soon, so we are already working on a proxy-based solution. Stay tuned!

Polkadot chain data backup node

To be able to bootstrap a fully synchronized Polkadot node in a few minutes, we have a specific backup Polkadot node running. This node synchronizes the Polkadot chain and sends a backup of the chain every day to remote storage. It allows us to rapidly repair or bootstrap new Archipel Polkadot validator nodes in case of a problem or chain corruption.

Supervision and alerting

To be sure that our validators are operating correctly, we need a solid supervision stack with real-time alerting.

Infrastructure Supervision

Firstly we need to supervise the underlying infrastructure. So we deployed a private supervision stack, including Grafana, Prometheus, and Alertmanager. This stack provides us real-time infrastructure monitoring and alerting.

Polkadot and Archipel nodes supervision

Polkadot and Archipel nodes are the main components of our solution. So to be sure that they are operating correctly, we deployed a double supervision stack.

Firstly, they are supervised by Grafana, Prometheus, and Alertmanager stack.

Secondly, a private Telemetry Substrate instance with a specific Archipel Alerting Hack Component is deployed. The Archipel Alerting Component, directly plugged to the telemetry back-end, analyzes the Archipel state and alerts us immediately via a Telegram Bot if something goes wrong.

Testing on Kusama network

We have tested the Archipel solution operation for several months on Kusama Network. This live operation helped us to improve our solution and to release the new version of Archipel.

Why are we launching our own validators?

We want to apply this best practice to us: “Eat your own dog food”. That is why we are launching those 3 Polkadot validators. We want to determine the relevance of the software we have developed so far.

By setting up those validators, we also seek to remain independent freebooters and continue developing and improving high available infrastructure tooling.

Our implementations are open source and can benefit all PoS chains, other blockchains services or even normal services as a base layer or inspiration for new consensus-based high availability solutions.

Archipel Polkadot validators summary

To sum up, we have:

  • 23 nodes (14 Polkadot nodes, 9 Archipel nodes)
  • 9 Archipel nodes: 3 groups of 3 nodes
  • 3 different Archipel Polkadot validators
  • 6 Polkadot passive nodes ready to take relay if necessary
  • 4 external sentry nodes for more security
  • 1 backup node for periodic backups of Polkadot chain for fast node recovery

We are using :

  • 9 different physical DAppNodes (Intel NUC hardware)
  • These nodes are distributed in 5 different locations, 5 cities, 2 countries, and 2 continents
  • 3 different cloud providers for public Sentry nodes
  • All connections between all nodes are secured with WireGuard VPN solution
  • All validators are highly-available thanks to Archipel
  • To prevent double signing, we are using the STONITH algorithm (SMS + 4G connected electric outlets)
  • A solid supervision stack with automatic alerting

If you want to support our project, and promote active purpose-driven development of validator infrastructure tooling, here are our validators details :

Archipel/Archipel Validator 1 16aFDVsp6zd6VxUSgd34es3r23nWRkoj3NdLTS5Fk1Ez9MU1

Archipel/Archipel Validator 2 16G6NN2PrePpCTy8DdMBTZdqcuLmWdrBDJNuYt4Q7EHKkSUR

Archipel/Archipel Validator 3 13Z21hBWPoTZH8J1wFzCyMefoQ976MRJzpSTvDGb2aqF3nmh

Our commission: 3%

You can find the Nominator Polkadot Wiki guide here and even more details in this video.

Disclaimer: Nominating validators provide rewards but also implies the risk of slashing. You can read more about staking and slashing in the Polkadot network here.

Archipel validators’ strategy tends to maximize the responsiveness, but we also take into consideration the equivocation risk. See non-responsiveness versus equivocation dilemma details here. For equivocation risk reduction (on blocks production and chain votes), Archipel orchestrator waits a defined number of blocks before switching validator nodes.

Stake at your own risk, we do not endorse any responsibility in case of capital loss!

Gaining rewards together, being slashed together*!

*If slashing happens our validators’ “own stake” and our nominators’ accounts will be also impacted.

#SkinInTheGame

Who are we ?

We are 2 open source develop-found-ers, at Lugus Labs with many years of experience in managing infrastructure and software in production.

Find us on twitter and on LinkedIn:

Welcome aboard Buccaneers!

Out of hegemonic, big cloud or staking providers !

--

--