Building an Ethereum playground with Docker (part 2 — Docker Image)

I have updated this article on December/2017. I no longer create a custom Docker image, because Ethereum official image already brings an “all-tools” version that includes bootnode and other treats.

This is the second article in an ongoing series “Building an Ethereum Playground with Docker”. The articles already published are:

• Building an E. P. Part 1 — Introduction
• Building an E. P. Part 2 — Docker Image (this one)
• Building an E. P. Part 3 — Ethereum Wallet
• Building an E. P. Part 4 — Provisioning

We will cover using the official “ethereum/go-ethereum” Docker image, playing with Ethereum Wallet, provisioning the ethereum nodes on public clouds and deployment of a sample app.

This article uses "ethereum/client-go" Docker image to run several Ethereum nodes locally (and safely). It also assumes you have a Docker engine available to you (and know a bit about it), probably after installing Docker for Mac or Docker for Windows on your notebook.

All sources are located in https://github.com/vertigobr/ethereum.git.

The Docker Image

There is a public official image "docker pull ethereum/client-go" that will serve as the base of this work. We will create scripts with some functionality and configuration options to make it generally useful for our evil machinations.

This original public base image is a nice piece of work: you can use it to participate on the main public Ethereum network with a simple command:

docker run -d --name ethereum-node \
    -v $HOME/.ethereum:/root \
    -p 8545:8545 -p 30303:30303 \
    ethereum/client-go:v1.7.3 --fast --cache=512

Or, similarly, to participate on the public test network:

docker run -d --name ethereum-node \
    -v $HOME/.ethereum:/root \
    -p 8545:8545 -p 30303:30303 \
    ethereum/client-go:v1.7.3 --testnet --fast --cache=512

To check the node logs:

docker logs -f ethereum-node

And to attach the the running node (main net) console:

docker exec -ti ethereum-node geth attach

Or to the running node on testnet:

docker exec -ti ethereum-node \
    geth attach ipc:/root/.ethereum/testnet/geth.ipc

But we won't be doing any of this — we want a private network of our own.

Getting Started

There is a Github project containing several helper scripts that will be used along this document:

git clone https://github.com/vertigobr/ethereum.git

Assuming you have Docker up and running:

• Run a bootnode
• Run a common node
• Run a miner node
• Check a node’s peers

That would be running the commands:

./bootnode.sh
./runnode.sh
./runminer.sh
./showpeers.sh

The starting point is the creation of a "genesis.json" file that defines the genesis block of the blockchain. The "genesis.sh" script does that for you, and it can be edited to provide custom values for some variables

Some variables on the top of this script can be modified to define your very own genesis block, the main point being that all containers will mount the same genesis.json file when launched with the helper scripts.

Nodes that share the same genesis block and are capable of finding each other (in our case using a bootnode to locate peers) will compose your private Ethereum network. Also the helper scripts mount data folders for each container, so if you stop and destroy the containers and run them again with the same arguments they will resume work as if nothing had happened.

The scripts

The project cloned from Github holds a few utility scripts we will use a lot to save time from typing long docker commands:

• genesis.sh: creates a new "genesis.json" file based on the template at "src/genesis.json.template" and on a few variables you can mess around with;
• bootnode.sh: runs an Ethereum bootnode in a container named "ethereum-bootnode";
• runnode.sh: runs an Ethereum non-mining node in a container named from your argument (ex: `runnode.sh node1` runs an `ethereum-node1` container);
• runminer.sh: identical to `runnnode.sh`, but starts a mining node;
• showpeers.sh: shows all peers that are connected to this node. Example: `showpeers.sh ethereum-node1`;
• killall.sh: "docker stop" and "docker rm" on all containers you ran with `runnode.sh` and `bootnode.sh`, but preserves the volume folders;
• wipeall.sh: “docker stop” and “docker rm” on all containers you ran with `runnode.sh` and `bootnode.sh` and also removes the volume folders.

All these scripts rely on the genesis.json file generated by "genesis.sh". When running them locally there is zero chance your network will chat with another Ethereum network, because everyone is restricted to the peers that found your bootnode *and* because your nodes are the only ones that can use your blockchain.

A few scripts deserve more explanation:

bootnode.sh

This script does the job below (reduced for simplicity):

docker stop ethereum-bootnode
docker rm ethereum-bootnode
docker run -d --name ethereum-bootnode \
 -v $(pwd)/.bootnode:/opt/bootnode \
 ethereum/client-go:alltools-v1.7.3 bootnode --nodekey (...)

This script starts a dumb bootnode instead of a "full" geth node. Also notice the local volume mounted on ".bootnode".

runnode.sh

This script does the job below (reduced for simplicity):

NODE_NAME=$1
CONTAINER_NAME="ethereum-$NODE_NAME"
docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME
BOOTNODE_URL=$(./getbootnodeurl.sh)
docker run -d --name $CONTAINER_NAME \
    -v $(pwd)/genesis.json:/opt/genesis.json \
    ethereum/client-go:v1.7.3 --cache=512 --bootnodes=(...)

Notice that the bootnode URL is detected with the utility `getbootnode.sh` and both container name and local volume are created based on the node name provided as an argument.

showpeers.sh

This script does the job below (reduced for simplicity):

docker exec -ti "$1" geth --exec 'admin.peers' attach

Notice the use of "docker exec" to submit a command to a current running node with "geth attach".

Let us Play

These are the steps we will follow:

1. Create the docker network
2. Run the bootnode;
3. Run a non-mining node;
4. Run a second non-mining node;
5. Check if both found their peer;
6. Run a mining node;
7. Check if everyone found their peers.

It is assumed, of course, that you have cloned the repository and have these scripts in your current folder. Once again, you don't need to build the image, Docker will pull it for you.

Here we go.

Step 1: Create network and run bootnode

This is very simple:

./bootnode.sh

You can check the container log:

docker logs ethereum-bootnode

It will result on something like this:

INFO [12-06|17:31:44] UDP listener up                          self=enode://d92e0ce77861919c516d8e6a65f58e441df0ec73b640551f3dd5e83c2e6bf41aa189e7709f261e633db0f906fe9b1e37c92eeb9d80b42918cb240726078439e3@[::]:30301

Notice that finding the bootnode from another container is possible in several ways, but I have opted to a low-level trick to detect the container internal IP address — this is an useless IP outside your engine (or swarm), but is good enough for us right now. Check, just to be sure, if the `getbootnode.sh` script returns a valid IP properly:

./getbootnode.sh

It will result on something like this (the actual IP may differ, obviously):

enode://xxxxx...yyyy@172.18.0.2:30301

Step 2: Run a non-mining node

This is also very simple:

./runnode.sh node1

You can check the container log:

docker logs ethereum-node1

It will result on something like this:

INFO [12-06|17:38:34] Starting peer-to-peer node               instance=Geth/v1.7.3-stable/linux-amd64/go1.9.2
(...)
INFO [12-06|17:38:34] Starting P2P networking
INFO [12-06|17:38:36] UDP listener up                          self=enode://454bc00e461761e8b3c7aa22f85a713c5f5a9b8032f253c5eca96c24acd1cbcc7af3058047aee53128d673f060c0a236b77c5da1947158566e5dd0cc4741a239@[::]:30303
INFO [12-06|17:38:36] RLPx listener up                         self=enode://454bc00e461761e8b3c7aa22f85a713c5f5a9b8032f253c5eca96c24acd1cbcc7af3058047aee53128d673f060c0a236b77c5da1947158566e5dd0cc4741a239@[::]:30303
INFO [12-06|17:38:36] IPC endpoint opened: /root/.ethereum/geth.ipc
(...)

The node feels lonely and keeps "dialing" to the bootnode to find other peers. Apart from that, it is quite useless.

You can change the verbosity argument on runnode.sh script to any value from 0 (silent) up to 6 (detail). Our script default is 4 (core messages).

Step 3: Run a second non-mining node

This is also very simple, hopefully:

./runnode.sh node2

You can check the container log:

docker logs ethereum-node2

Notice that via bootnode the first node will be found by this new node. You now have an Ethereum network with two peers that found each other. You can easily check back the first container log:

docker logs ethereum-node1

You will find content like this:

DEBUG[12-06|18:02:56] Adding p2p peer                          id=0258936c1c6ede4b name=Geth/v1.7.3-stable/l... addr=172.18.0.4:51936 peers=1
DEBUG[12-06|18:02:56] Ethereum peer connected                  id=0258936c1c6ede4b conn=inbound name=Geth/v1.7.3-stable/linux-amd64/go1.9.2

Step 4: Check if both found their peer

It is a lot easier than peeking logs to use the `showpeers.sh` script:

./showpeers.sh node1

This returns more meaningful content for each peer this node is connected to (remember, the bootnode does not count):

[{
    caps: ["eth/63"],
    id: "06055abb9386a74596cc0486430abfc3b967d9bba643bed799e8666e5caff3661c9eaa639a98f5c7c235ec003070dd0792d5838505252b4b5ce83d90a451e77b",
    name: "Geth/v1.7.3-stable/linux-amd64/go1.9.2",
    network: {
      localAddress: "172.18.0.3:30303",
      remoteAddress: "172.18.0.4:51984"
    },
    protocols: {
      eth: {
        difficulty: 131072,
        head: "0x2084e6ce93b07f22db9e3e5960b5911fd493214d886161c49e1a00753052b987",
        version: 63
      }
    }
}]

Same kind of output you will get by checking the other node:

./showpeers.sh node2

Congratulations, you already have a private Ethereum network in your hands!

Step 5: Run a mining node

A non-mining blockchain kinda makes no sense, so we will spin a mining node to make it work (and just for the fun of it).

./runminer.sh miner1

It will take quite some time until the mining node begins mining. You will find this slow progression on the logs:

docker logs -f ethereum-miner1
...
INFO [12-06|18:08:53] Generating DAG in progress               epoch=0 percentage=0 elapsed=3.893s
INFO [12-06|18:08:57] Generating DAG in progress               epoch=0 percentage=1 elapsed=7.695s
...

After that you will begin to see successful mining logs:

INFO [12-06|18:16:06] 🔗 block reached canonical chain          number=10 hash=90370f…e58ba1
INFO [12-06|18:16:06] 🔨 mined potential block                  number=15 hash=44b3de…25e7d9
INFO [12-06|18:16:06] Commit new mining work                   number=16 txs=0 uncles=0 elapsed=230.484µs
DEBUG[12-06|18:16:06] Reinjecting stale transactions           count=0
DEBUG[12-06|18:16:06] Recalculated downloader QoS values       rtt=20s confidence=1.000 ttl=1m0s
INFO [12-06|18:16:10] Successfully sealed new block            number=16 hash=b171fe…003c8f

The non-mining nodes will get copies of mined blocks, as they are expected to. Just check back the first node log:

docker logs ethereum-node1

And you will see lines like these:

DEBUG[12-06|18:16:33] Queued propagated block                  peer=06055abb9386a745 number=22 hash=f8e9b2…5cfb00 queued=1
DEBUG[12-06|18:16:33] Importing propagated block               peer=06055abb9386a745 number=22 hash=f8e9b2…5cfb00
DEBUG[12-06|18:16:33] Trie cache stats after commit            misses=7 unloads=0
DEBUG[12-06|18:16:33] Inserted new block                       number=22 hash=f8e9b2…5cfb00 uncles=0 txs=0 gas=0 elapsed=13.009ms

Step 6: Check if everyone found their peers

It is quite obvious that all nodes found the other peers (or the mining block would not propagate). You can easily check if it out anyway:

./showpeers.sh miner1

A response like the one below is expected:

[{
    caps: ["eth/63"],
    id: "06055abb9386a74596cc0486430abfc3b967d9bba643bed799e8666e5caff3661c9eaa639a98f5c7c235ec003070dd0792d5838505252b4b5ce83d90a451e77b",
    name: "Geth/v1.7.3-stable/linux-amd64/go1.9.2",
    network: {
      localAddress: "172.18.0.5:41094",
      remoteAddress: "172.18.0.4:30303"
    },
    protocols: {
      eth: {
        difficulty: 3820928,
        head: "0x0ba0118d469b82875abfc6706de5d2463b6519fd8a71a63d939834cc14fd4ebd",
        version: 63
      }
    }
}, {
    caps: ["eth/63"],
    id: "204a74e1251671597e16207675a78853b24fddad96a795c1a2a14717a12deabcb440d2bcce9e5c6066af6f86a9c2e327ded161d9f9ebf01b0bc64fd2ed3100b0",
    name: "Geth/v1.7.3-stable/linux-amd64/go1.9.2",
    network: {
      localAddress: "172.18.0.5:30303",
      remoteAddress: "172.18.0.3:56028"
    },
    protocols: {
      eth: {
        difficulty: 131072,
        head: "0x2084e6ce93b07f22db9e3e5960b5911fd493214d886161c49e1a00753052b987",
        version: 63
      }
    }
}]

I Wanna Do It Again

Why not? After all these are containers, not VMs. You can stop an destroy them all:

./killall.sh

Running the scripts again with the same arguments will "resurrect" your network (the mounted volumes are still there):

./bootnode.sh
./runnode.sh node1
./runnode.sh node2
./runminer.sh miner1

The mined blocks are still there, of course — all pertinent state lives on the mounted volumes — so this second run is a lot faster.

Wanna really restart from scratch?

./wipeall.sh

This script also empties the mounted volumes.