Polkadot: A Syncing Node in under 30s

Note: Read this article in German on Base58.de.

Using the new Polkadot Docker image to get started with PoC-2.


Exciting times

The Polkadot PoC testnet has been up and running for a while, but for newcomers who want to jump in the level of technical detail may seem a little daunting. Don’t let that stop you!

The Polkadot node software is developed using Rust, a very modern, fast and robust programming language.

In the past, ParityTech showed a pleasant tendency to open-source its software in very early stages. Once again with Polkadot, the code has been made available to everyone, allowing contributors and testers to participate in the network’s evolution from the start.

As you can see in the very first commit, Polkadot began with a simple “Hello, world!” and ReadMe from Robert Habermeier.

Source: GitHub

Whether a developer or a user, the first thing anyone interested in Polkadot should do is get the network running. Let’s dive into how we can do that.


Building Polkadot without Docker

In the early days of such a project, binary installers for Linux, OS X or Windows are usually not available (you can find the Polkadot releases here) so the only way to use them is to build the software yourself.

Although the process is well documented, building a project such as Polkadot may not be trivial unless you are a Rust developer or plan on become one :).

Source: GitHub

Assuming the instructions above make sense to you, you will still need some patience (and fan-spinning…) before you see your node run and show up in the Polkadot telemetry site.

Compiling software such as Polkadot can take from 30 minutes to a few hours depending on the computer available to you. There must be a faster way!
Source: Tim Gouw on Unsplash

Docker to the rescue

Docker can be seen as a very slim virtualisation solution. It is rather easy to install and you can find all the instructions here.

Docker will help us solve two problems:

  • Building Polkadot easily or even skipping this step
  • Deploying a Polkadot nodes in less than 30s*

*This is somewhat dependent on your internet connection.


Use a Docker pre-built image

So you are lazy? Me too!

Once you’ve installed Docker, the only thing you need is a shell to enter a few commands. How about a few one-liners to get it to just run and start syncing?

In theory, you should first pull the image you want to use but I mentioned how lazy I am, so we skip that. Docker will deal with it!

We start simple and will check what version of Polkadot we have.

You can copy/paste the command below, just make sure you exclude the “$”. It is just telling us this is a command to type into the shell. Linux users may also have to add sudo before each docker command.

$ docker run --rm -it chevdor/polkadot:0.2.0 polkadot --version

Before we look at the result, let’s have a look at what‘s going on. You wouldn’t blindly run unknown commands from a random article on the internet, right?

  • docker run well… does invoke docker and asks it to run a container.
  • --rm means everything related to the container will be deleted and cleaned up once we kill the container. This is good for now.
  • -it will show us what is going on inside this container
  • chevdor/polkadot:0.2.0is the name and version of the pre-built image we want to use. If you feel adventurous, you may even try chevdor/polkadot:latest.
  • polkadot --versionNothing you can invent here. You need to trust me on this one (and if you don’t, you can still read the code!). This will show you the Polkadot version and a hash. The Polkadot devs will be thankful if you report an issue and include this.

To find all the available versions, you can visit: https://hub.docker.com/r/chevdor/polkadot/tags/

Source: Hub.Docker.com

You should see something similar to the output below. Don’t let Docker turn you down with its Unable to find image, this is normal: it means docker does not find the image locally on your machine and needs to download it first.

$ docker run --rm -it chevdor/polkadot:0.2.0 polkadot --version
Unable to find image 'chevdor/polkadot:0.2.0' locally
0.2.0: Pulling from chevdor/polkadot
Digest: sha256:59bacb0e77e32690fd0c3...69d0d1563ef1adb
Status: Downloaded newer image for chevdor/polkadot:0.2.0
polkadot 0.2.0

How about running a real node now? Notice the small change, we remove--version :

$ docker run --rm -it chevdor/polkadot:0.2.0 polkadot
2018–07–23 17:09:02 Parity ·:· Polkadot
2018–07–23 17:09:02 version 0.2.0
2018–07–23 17:09:02 by Parity Technologies, 2017, 2018
2018–07–23 17:09:02 Chain specification: Krumme Lanke
2018–07–23 17:09:02 Node name: glamorous-whistle-6100
2018–07–23 17:09:02 Starting (heavy)
2018–07–23 17:09:02 Generated a new keypair: cc41c8ec25...d2a6b (5GgXAgp2…)
2018–07–23 17:09:03 Initialising Genesis block/state (state: 609a…3f5a, header-hash: 6c5a…71cc)
2018–07–23 17:09:03 Best block: #0
2018–07–23 17:09:03 Reconnected to telemetry server: wss://telemetry.polkadot.io/submit/
2018–07–23 17:09:03 Local node address is: /ip4/0.0.0.0/tcp/30333/p2p/Qmbr2BxgdUTzB6CWf...XY3dksNos8rcciu5N
2018–07–23 17:09:03 Idle (0 peers), best: #0 (6c5a…71cc)

Now we can start having fun because the polkadot you see at the end of the command *is* the real polkadot. Let’s ask it for some help:

$ docker run --rm -it chevdor/polkadot:0.2.0 polkadot --help
polkadot 0.2.0
Parity Team <admin@parity.io>
Polkadot Node Rust Implementation
USAGE:
polkadot [FLAGS] [OPTIONS] [SUBCOMMAND]
FLAGS:
--dev             Run in development mode; implies --chain=dev --validator --key Alice
-h, --help Prints help information
--light Run in light client mode
<...>

Optionally, if you are using Linux or OS-X, you can use an alias and forget you are using docker. In the following example, I named the alias polkadocker but you may name it anything.

$ alias polkadocker=’docker run --rm -it chevdor/polkadot:0.2.0 polkadot’
$ polkadocker --help
polkadot 0.2.0
Parity Team <admin@parity.io>
Polkadot Node Rust Implementation
USAGE:
polkadot [FLAGS] [OPTIONS] [SUBCOMMAND]
<...>

Very cool, but we need to improve a bit more to properly run a node:

  • open the required ports
  • mount a volume

Opening the correct ports for Polkadot is important if you want to communicate with your node. See the -p flags below.

Above all, mounting a volume is super important. This is the -v you see below. If you don’t do that, all the data, including those blocks you synchronised the last hours/days will be gone once you stop the container or if for whatever reason it crashes.

$ mkdir -p /home/<youruser>/polkadot-data
$ docker run --rm -it -p 30333:30333 -p 9933:9933 -v /home/<youruser>/polkadot-data/:/data chevdor/polkadot:latest polkadot

Now our node will store all the important data in /home/<youruser>/polkadot-data. You can now stop your container, start it again, it will resume where it previously was.

If you are also using a local Polkadot binary, you may want to use the following folders:

  • OS-X: ~/Library/Application Support/Polkadot/
  • Linux: ~/.local/share/Polkadot/

Picking a cool name

If you don’t provide a name to your node, Polkadot will find one for you (sometimes with a funny touch, give it a shot). You can however name your node using --name "<name here>"

$ docker run --rm -it -p 30333:30333 -p 9933:9933 -v /home/<youruser>/polkadot-data/:/data chevdor/polkadot:latest polkadot --name "jumpy-monkey-4448"

Almost there! At that point, we are all good but closing the shell where docker runs will stop the container :(.


Running as daemon

Instead of entertaining us with scrolling logs, Docker can run in the background and this is super simple with -d:

$ docker run -d -p 30333:30333 -p 9933:9933 -v /home/<youruser>/polkadot-data/:/data chevdor/polkadot:latest polkadot

You can now check telemetry.polkadot.io, your node should show up.

https://telemetry.polkadot.io/

Congratulations!

If you followed along so far, you should see something similar to the console capture below and you should be syncing already.

Source: https://asciinema.org

You are now part of a unique experiment!

Be proud and show-off shamelessly on Twitter. Mention @polkadotnetwork and @chevdor if this article helped you get started.


About the author

Will is also writing article for Base58.de where Blockchain news are decrypted and explained in German. Find this article translated to German at https://base58.de/polkadot-knoten-in-unter-30-sekunden-erstellen

References & links