Building an Ethereum playground with Docker (part 4— Provisioning)

20 min readSep 17, 2016

I have updated this article on January/2018. I no longer create a custom Docker image, because Ethereum official image already brings an “all-tools” version that includes bootnode and other treats.

This is the fourth article in a series on “Building an Ethereum Playground…”. The articles already published are:

Building an E. P. Part 1 — Introduction
Building an E. P. Part 2 — Docker Image
Building an E. P. Part 3 — Ethereum Wallet
Building an E. P. Part 4 — Provisioning (this one)

We will cover using the official “ethereum/go-ethereum” Docker image, playing with Ethereum Wallet, provisioning the ethereum nodes on public clouds and deployment of a sample app.

All sources and scripts are located in https://github.com/vertigobr/ethereum.git.

This article is about the art of provisioning servers on public cloud providers in a way that you forget about them as soon as possible. We will always pursue a "CaaS" (Container-as-a-Service) workflow — in real life this results on portable projects, immutable infrastructure and better sleep time.

The Point of CaaS and Serverless

We will discuss in this article how to provision a a cluster of container nodes to run our private Ethereum, wether locally (using Docker for Mac or Docker for Windows) or on public cloud providers like Digital Ocean.

Containers are incredibly useful in many different situations — they fit many needs for developer workstations, CI/CD pipelines and software packaging and deployment.

Provisioning a server to run a few containers, on the other hand, is pretty much "same old same old": you get your hands on a VM, install the Docker engine and "docker run" whatever you want. You know where your application runs and you are expected to maintain this server/service running.

A better workflow for developers (and modern IT) is proposed by CaaS (Container as a Service) and, more generically, serverless approaches. This strategy works in a way where the developer does not care where the application runs — as long as it runs in a predictable way and the path from code to deployment is as simple as possible.

Docker on the workstation

On Windows machines Docker can be installed in two ways:

Docker for Windows (recommended): for earlier Windows versions (like Windows 10 Pro);
Docker Toolbox: for other/older Windows versions.

Note: in both cases Windows users will be dealing with a Linux 64-bit virtual machine. Docker Toolbox uses an "explicit" VirtualBox VM, Docker for Windows relies on a "hidden" Hyper-V VM.

On Macs you can also install Docker in two ways:

Docker for Mac (recommended): for earlier OSX versions;
Docker Toolbox: for older OSX versions.

Note: in both cases Mac users will be dealing with a Linux 64-bit virtual machine. Docker Toolbox uses an “explicit” VirtualBox VM, Docker for Mac relies on a “hidden” Hyperkit VM. At the current moment there is no native Docker Engine for OSX.

On Linux workstations you can install Docker natively, but there are usually two choices:

Install Docker using the native Linux distro version;
Install Docker from Docker Inc.’s distro.

Either way on Linux you will have to install manually the client tools "docker-compose" and "docker-machine", because they are not included on the distro installations. Instructions can be found easily on Docker site.

Some Linux distros have lagged behind a lot on Docker releases (or are too busy making a “Frankendocker” like Red Hat/CentOS), so if you want to use Docker’s latest features you’ll always prefer Docker’s Docker CE (or EE) release.

There are a few other options to install Docker locally on Windows or Mac systems, if the above options present problems:

You can use the "docker-machine" tool to provision a local VirtualBox VM;
You can use Vagrant and provision a local VM based on a Docker-enabled box.

Your local tools

Even if you plan not to run Docker locally, you will want to install Docker client tools locally in your workstation in order to deploy/control Docker nodes remotely (we will get into that later on).

This document assumes you are working in a shell where:

Docker client is available (ex: a plain terminal after Docker for Mac/Windows or Docker Toolbox is installed);
You have Git installed *and* you have cloned the sample repository;
You are capable of generating an SSH key (you can do it with Putty in Windows). C’mon, you did it when you created your Github account;
You have a few unix tools in the command-line (specially “ssh”) — Windows users have a few options nowadays, like Cygwin or Putty for older Windows versions or "Windows Subsystem for Linux" for Windows 10/Windows Server. There is even a current native OpenSSH (beta) for Windows in the works.

Docker on the Cloud

There are several options to experiment Docker for free (or for a very small cost) on the cloud. Most cloud providers give away some credit that can be used to provision the resources you need to run containers, but there are so many options around these days that people can get confused. So we are focusing only on those that result in a Docker API available at the user's command line, thus avoiding other proprietary approaches.

Let us get some background on a few public cloud provider tools. You can get some freebie time ou beta access on most of them, so it is assumed you’ve already done that and have your logins, keys and/or subscription ids at hand (each provider has its own authentication waltz).

Digital Ocean: a developer's favorite, simple and fast cloud provider. Fair price, no hidden costs;
AWS: Amazon cloud provider, the first big player, market leader by far, multiple offerings;
Azure: Microsoft's cloud provider, big player, multiple offerings;
GCP: Google's cloud provider, big player, multiple offerings;
Bluemix: IBM's cloud provider, big player, multiple offerings;

We are going to provision clusters in:

Docker Machine (Digital Ocean example)
Azure (todo)
GCP (todo)

The “setenv.sh” script

This document assumes you have a local `setenv.sh` script that sets the proper environment variables to whatever values are needed to connect to your cloud provider.

A template to such a file is present on the repository, named `setenv.sh.template`. Copy it into `setenv.sh` and modify its variables to what your provider requires:

Digital Ocean: requires environment variable DIGITALOCEAN_ACCESS_TOKEN;
Azure on docker-machine: needs a valid subscription id from Azure;

Docker Machine (Digital Ocean)

The "docker-machine" tool is a command-line utility that simplifies provisioning cloud (or local) VMs with Docker pre-installed.

$ docker-machine create -d digitalocean eth-node1
Running pre-create checks...
Creating machine...
(...)
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env eth-node1

The "docker-machine" tool can be used to log into the new VM (something you actually want to avoid):

$ docker-machine ssh eth-node1
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-103-generic x86_64)
(...)

The best possible workflow is to deal with the remote docker engine directly, since the VM was provisioned exposing the Docker TCP port in a secure way (TLS). To configure your docker command line to talk to the remote engine just type:

eval $(docker-machine env eth-node1)

Check the docker client and engine version (they will probably differ), also validating the connection:

$ docker version
Client:
 Version: 17.12.0-ce-rc4
 API version: 1.35
 Go version: go1.9.2
 Git commit: 6a2c058
 Built: Wed Dec 20 15:53:52 2017
 OS/Arch: darwin/amd64Server:
 Engine:
  Version: 17.12.0-ce
  API version: 1.35 (minimum version 1.12)
  Go version: go1.9.2
  Git commit: c97c6d6
  Built: Wed Dec 27 20:09:53 2017
  OS/Arch: linux/amd64
  Experimental: false

Hey, You Are Working With a Remote Engine

It is great to work directly with a remote engine in the cloud from your command line — you gradually adapt your mindset to a serverless mindset.

All "docker" commands will run on the context of this remote server, so remember a few things:

A "docker pull" will pull images into this remote server;
A "docker run" will start a container in this remote server;
A "docker build" will work uploading files (the build context) into this server;
A "docker run" that mounts local volumes with "-v" will mount volumes on the remote server;
…and so on.

In all scenarios here where a VM is created we will favor using passwordless SSH login (i.e. private/public SSHkeys).

I am one of those who consider the term "serverless" a very confusing and silly one (or at least quite a poor choice). No software runs on ether (not even Ethereum) — there is always a server somewhere that costs money and suffers from bad code (deal with it).

The philosophy of serverless, on the other hand, is a smart one: you don't need to care about server provisioning and may outsource (or ignore) it completely. Martin Fowler wrote a good piece about it: serverless on the real world materializes on offers on BaaS and FaaS space. This is a very rich subject — and also a sketchy one, for some FaaS choices put us back in the old vendor lock-in trap.

Back to containers, though, the serverless philosophy comes forward on CaaS (Container as a Service) offerings, a generic term for a deployment strategy where you pretty much ignore where your container will run. As Docker evolved several orchestration options rose to market (Kubernetes and Swarm mode, for example), where a cluster of servers allow a CaaS approach.

Provisioning a container infrastructure in your development machine in the old days was done with Boot2Docker, later on with Docker Toolbox and (more recently) with Docker for Mac/Windows. Linux workstations, on the other hand, may install Docker natively.

When provisioning Linux servers on public cloud providers you can usually count on the cloud provider's GUI or command-line tools, or you can rely on generic automation tools (like terraform). Either way you end up with Linux VMs where Docker yet needs to be installed.

Several cloud providers will offer higher-level choices: Docker CE/EE templates, Kubernetes managed offerings, the list goes on and on.

Alas in all cases you should be pursuing a "CaaS lifestyle" — reaching a workflow where your "docker" commands talk to a local or remote infrastructure the same way.

In other words, you want to ignore the VMs existence whenever you can.

So this article will tread on a few provisioning choices for a generic container-based clustered deployment where, at the end, the reader will be able to start, stop, wipe out and fool around with its nodes the same way, regardless of how/where/by whom they were provisioned.

Docker on the Cloud

Let us get some background on a few public cloud provider tools. You can get some freebie time ou beta access on pretty much all of them, so it is assumed you've already done that and have your logins, keys and/or subscription ids at hand (each provider has its own authentication waltz).

We are going to provision clusters in:

Digital Ocean (with docker-machine)
(todo)

Your local tools

This document assumes you are currently in a shell where:

Docker client is available (like in an OS prompt after Docker for Mac/Windows is installed);
You have cloned this sample repository and "cd" into it;
You are capable of generating an SSH key (you can do it with Putty in Windows). C'mon, you did it when you created your Github account;
You have a few unix tools in the command-line (like "ssh") — Windows users could try a few options like Cygwin, or work around with tools like Putty.

In all scenarios here where a VM is created we will favor using passwordless SSH login (i.e. private/public SSHkeys).

The “setenv.sh” script

This document assumes you have a local `setenv.sh` script that sets several environment variables to whatever values are needed to connect to your cloud provider.

A template to such a file is present on the repository, named `setenv.sh.template`. Copy it into `setenv.sh` and modify it to what your provider requires:

Digital Ocean: requires environment variable DIGITALOCEAN_ACCESS_TOKEN;
Azure on docker-machine: needs a valid subscription id from Azure;
IMPORTANT: You should change names that would conflict with any other resources on Azure cloud (ex: the "AZURE_DNS_PREFIX" variable that names the VMs hosts created on Azure).

Digital Ocean

This gives us a new swarm (a cluster of Docker engines) name "ethereum" and ready to roll in a very short time. You can always check it out on getcarina.com web console:

At any time you can configure your current shell to use this swarm as the target for `docker` commands:

eval $(carina env ethereum)

This command sets several environment variables recognized by the Docker client. Check these with the command below:

➜  env | grep DOCKER
DOCKER_HOST=tcp://146.20.69.138:2376
DOCKER_TLS_VERIFY=1
DOCKER_CERT_PATH=(...)/.carina/clusters/xxx@yyy/ethereum
DOCKER_VERSION=1.11.2

The exposed engine API port (2376) is useless unless you have its keys, so this is a safe setup. This is actually what Docker recommends.

The resulting behavior is that all docker commands are now sent to Carina (and not to your local engine). On my Mac, for example, `docker version` shows these results (notice the difference between the client and the engine):

➜  docker version
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 17:32:24 2016
 OS/Arch:      darwin/amd64
 Experimental: trueServer:
 Version:      swarm/1.2.3
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   eaa53c7
 Built:        Fri May 27 17:25:03 UTC 2016
 OS/Arch:      linux/amd64

Congratulations, you are now living on CaaS style. Ideally you should be able to run containers the same way you did before — the fact that the engine is somewhere else should be irrelevant. We'll get back to it later.

Notice: at the time of this writing the Carina swarm is not an implementation of the new swarm mode of Docker 1.12 (i.e. no "docker service" commands). Instead Carina swarm works in a way that your "docker run" commands are attended by any node in the swarm. Rackspace states that the new swarm mode will be supported in the future.

Azure Container Service (Azure CLI)

Azure is Microsoft's cloud provider that competes (or tries to) with AWS. It is a generic cloud provider (not focused solely on containers like Carina is), but it has some VM templates aimed at Docker users.

One way to create a swarm cluster is using Azure CLI and a recently available template for that: the Azure Container Service. You can follow this link to create the ACS swarm manually or try the semi-scripted path below.

Please note that, at the end of the day, with ACS you get a somewhat cumbersome experience and still have to deal with VMs — this is nothing at all a CaaS workflow, but we will get as close as possible.

Download and install instructions for Azure CLI are here.

First you have to login into Azure with `azure login`:

➜  azure login
info:    Executing command login
/info:    To sign in, use a web browser to open the page https://aka.ms/devicelogin. Enter the code XXXXXXXXX to authenticate.

Azure CLI authentication asks you to validate this login from any device with a browser. Do as you are told and the command above will continue:

/info:    Added subscription Visual Studio Enterprise
info:    Added subscription Azure Pass
info:    Setting subscription "Visual Studio Enterprise" as default
+
info:    login command OK

Test the current session with the command below:

azure account show

Now try the command:

➜ . ./setenv.sh
➜ azure config mode arm
info:    Executing command config mode
info:    New mode is arm
info:    config mode command OK
➜ azure group create $AZURE_RESOURCE_GROUP $AZURE_LOCATION
(...)
info:    group create command OK
➜ azure group deployment create $AZURE_RESOURCE_GROUP $AZURE_DEPLOYMENT_NAME --template-uri $AZURE_TEMPLATE_URL -e azure/azuredeploy.parameters.json

The ACS will then be created:

info:    Executing command group deployment create
+ Initializing template configurations and parameters
+ Creating a deployment
...
data:    Outputs            :
data:    Name        Type    Value
data:    ----------  ------  --------------------------------------------------------------------------
data:    masterFQDN  String  ethereum-playgroundmgmt.eastus.cloudapp.azure.com
data:    sshMaster0  String  ssh etheruser@ethereum-playgroundmgmt.eastus.cloudapp.azure.com -A -p 2200
data:    agentFQDN   String  ethereum-playgroundagents.eastus.cloudapp.azure.com
info:    group deployment create command OK

Our template creates a Swarm with one master node and three agent nodes (this is the "old" Docker Swarm, not the Swarm mode from Docker 1.12.1).

Azure takes quite some time to finish, but eventually you will be able to see on its web admin console all the resources created under "EthereumPlayground" resource group.

IMPORTANT: our script created all these VMs as Ubuntu 14.04 with a local (sudoer) user named "etheruser". Also this user accepts key-based SSH logins with your default "id_rsa" key (your public key "id_rsa.pub" was used to configure the VMs).

You can — but you usually WON'T — access your VMs with SSH (port 2200). Your default private key would work (passwordless login):

➜ ssh etheruser@ethereum-playgroundmgmt.eastus.cloudapp.azure.com -p 2200

Now the closest we can get to a CaaS lifestyle on ACS (where you issue “docker” commands in your workstation talking to the remote swarm master) is achieved creating a SSH tunnel to the master Engine API port. Yes, it feels weird, but that's what Microsoft recommends. You create the tunnel and set DOCKER_HOST variable to use it:

➜ ssh -L 3375:localhost:2375 -f -N etheruser@ethereum-playgroundmgmt.eastus.cloudapp.azure.com -p 2200
➜ export DOCKER_HOST=localhost:3375
➜ docker ps
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 17:32:24 2016
 OS/Arch:      darwin/amd64
 Experimental: trueServer:
 Version:      swarm/1.1.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   a0fd82b
 Built:        Thu Feb  4 08:55:18 UTC 2016
 OS/Arch:      linux/amd64

There you go, your swarm cluster is online.

Notice: at the time of this writing, the Docker swarm created this way is the old version of Swarm (usually refered as "standalone Swarm" on Docker docs), not the new "Swarm mode" introduced in Docker 1.12.

Azure on Docker-Machine

Another way to provision a Docker Swarm on Azure is using the `docker-machine` utility, a tool provided by Docker to simplify the provisioning of Docker engines/clusters on any cloud provider its drivers know about.

The `docker-machine` workflow delegates to its drivers all the hassle of dealing with cloud vendor specific mumbo-jumbo.

Notice: at the time of this writing, docker-machine can indeed be used to provision a swarm with its "--swarm" arguments but the Docker swarm created this way is the old version of Swarm (usually refered as “standalone Swarm” on Docker docs), not the new “Swarm mode” introduced in Docker 1.12. We are going to provision a cluster with the new Swarm mode — so we will use docker-machine just to provision simple Docker nodes and later on we will turn them into a swarm.

To provision a simple Docker node (i.e. an Azure VM with Docker pre-installed) on Azure we run :

➜ source setenv.sh
➜ docker-machine create --driver azure \
  --azure-subscription-id $AZUREDM_SUBSCRIPTION_ID \
  --azure-docker-port 2376 \
  --azure-location $AZUREDM_LOCATION \
  --azure-resource-group $AZUREDM_RESOURCE_GROUP \
  --azure-ssh-user etheruser \
  docker-node1
Running pre-create checks...
(docker-node1) Completed machine pre-create checks.
Creating machine...
...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env docker-node1

Notice: Azure authentication will hang your command-line and ask you to open a specific URL in a browser on any device available and complete a web login. This will surely happen on the first time you run docker-machine, but authentication is cached and won't bother you again for some time.

Repeat the command above to create nodes `docker-node2` and `docker-node3`. At the time of this writing the VMs created on Azure this way are running Ubuntu 16.04 LTS and Docker 1.12.

After creating all three VMs you can list them with the command below:

➜  docker-machine ls
NAME           ACTIVE   DRIVER   STATE     URL   SWARM   DOCKER...
docker-node1            azure    Timeout
docker-node2            azure    Timeout
docker-node3            azure    Timeout

To set your local shell to work with the remote "docker-node1" engine you just have to run:

➜ eval $(docker-machine env docker-node1)

This sets a bunch of environment variables that affect your Docker client. To test the connectivity with the remote Engine run `docker version`:

➜  docker version
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.7.1
 Git commit:   6f9534c
 Built:        Thu Sep  8 10:31:18 2016
 OS/Arch:      darwin/amd64Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Each of these Azure VMs has a public IP (a "real" IP visible to the world) and a private IP (visible to other VMs in the same Azure virtual network). To find `docker-node1` public IP just type:

➜ docker-machine ip docker-node1

And to find its private IP just type (look for the eth0 device):

➜ docker-machine ssh docker-node1 ip addr
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether XX:XX:XX:XX:XX:XX brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.4/16 brd 192.168.255.255 scope global eth0
...

Ok, now it is time to "convert" these three independent Docker engines into a swarm (the new Swarm mode from Docker 1.12). We will start with `docker-node1` and move to each node afterwards (please replace "192.168.0.4" with your own private IP discovered as above explained).

➜ eval $(docker-machine env docker-node1)
➜ docker swarm init --advertise-addr 192.168.0.4
Swarm initialized: current node (23coeyfn8tsexzlvcehxfpjac) is now a manager.To add a worker to this swarm, run the following command:docker swarm join \
    --token SWMTKN-1-20m9v...2y68rl \
    192.168.0.4:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions..

Easy! Write down the join snippet from the output (specially the token string), we will run it for each node. To check the swarm members:

➜  docker node ls
ID              HOSTNAME      STATUS  AVAILABILITY  MANAGER STATUS
23co...fpjac *  docker-node1  Ready   Active        Leader

To add `docker-node2` to the swarm just run the command below:

➜ eval $(docker-machine env docker-node2)
➜ docker swarm join \
    --token SWMTKN-1-20m9vvf...2y68rl \
    192.168.0.4:2377
This node joined a swarm as a worker.

Repeat for `docker-node3` and you will have the swarm ready:

➜ eval $(docker-machine env docker-node1)
➜  docker node ls
ID             HOSTNAME      STATUS  AVAILABILITY  MANAGER STATUS
0wz4...c2i2    docker-node3  Ready   Active
23co...pjac *  docker-node1  Ready   Active        Leader
9s5k...scke    docker-node2  Ready   Active

Amazon on ECS-CLI

A cluster of Docker engines can be provisioned on AWS with a fast-food tool called `ecs-cli` that can be downloaded here. Configuring ECS-CLI is simple:

➜ # sets env vars
➜ source setenv.sh
➜ ecs-cli configure --region=$AWS_REGION --cluster $AWS_CLUSTER
INFO[0000] Saved ECS CLI configuration for cluster (ethereum-playground)

Assuming you already created a keypair in AWS console, a 3-node cluster is created in a simple instruction:

➜ ecs-cli up --keypair id_rsa --capability-iam --size 3 --instance-type t2.nano
INFO[0002] Created cluster                               cluster=ethereum-playground
INFO[0005] Waiting for your cluster resources to be created
...

Amazon on docker-machine

As usual, docker-machine keeps it simple. To create a Docker-enabled VM on AWS you can type:

➜ # sets env vars
➜ source setenv.sh
➜ docker-machine create --driver amazonec2 \
  --amazonec2-region $AWS_REGION \
  --amazonec2-zone b \
  --amazonec2-instance-type t2.micro \
  --amazonec2-ami ami-c60b90d1 \
  etherplay-1
Running pre-create checks...
Creating machine...
(etherplay-1) Launching instance...
...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env etherplay-1

Notice: the "--amazonec2-zone" is "b" because the default "a" is a non-existant (or non-available) zone, and the "--amazonec2-instance-type" is "t2.micro" because this is the instance that falls in the monthly free tier. The "--amazonec2-zone" is forcing a Ubuntu 16.04 LTS, because in my tests the default VM image was 15.01. If a future docker-machine release fixes this you may remove these arguments. BTW, AMI images per region can be found here.

Do the same for other VMs named "etherplay-2" and "etherplay-3". Just like in the Azure example we will configure these nodes into Swarm mode. We choose "etherplay-1" to become the swarm master, so we need to know its private IP (look for the "eth0" device):

➜ docker-machine ssh etherplay-1 ip addr
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 ...
    link/ether xx:xx:...:xx brd ff:ff:ff:ff:ff:ff
    inet 172.31.63.152/20 brd 172.31.63.255 scope global eth0
       valid_lft forever preferred_lft forever
...

We can now enable Swarm mode on "etherplay-1" (please use the IP you found):

➜ eval $(docker-machine env etherplay-1)
➜ docker swarm init --advertise-addr 172.31.63.152
Swarm initialized: current node (1qvglfulku1lxiz0o57vqnc2t) is now a manager.To add a worker to this swarm, run the following command:docker swarm join \
    --token SWMTKN-1-5aqjx...bp0wzs \
    172.31.63.152:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

Write down the token string to use it later. To add "etherplay-2" and "etherplay-3" as worker nodes (use the token string obtained on the last step):

➜ eval $(docker-machine env etherplay-2)
➜ docker swarm join \
    --token SWMTKN-1-5aqjxa...bp0wzs \
    172.31.63.152:2377

Finally check the swarm member list — the swarm is ready!

➜ eval $(docker-machine env etherplay-1)
➜ docker node ls
ID                     HOSTNAME     STATUS  AVAILABILITY  MANAGER..
16fidk7...jvddy34x0    etherplay-3  Ready   Active
3wdbip1...63mjmqen6 *  etherplay-1  Ready   Active        Leader
4tntm5f...2zce00rlo    etherplay-2  Ready   Active

Notice: for some reason during my tests port 2377 in was blocked on the private IP. In that case joining fails and `docker node ls` returns only the swarm master. To fix this you must edit the security group "docker-machine" in AWS console and enable an inbound rule (incoming port=2377, source=the security group itself).

Other Cloud Providers

Other cloud providers will behave the same way: there is a command-line tool that can be used to provision VMs based on well-known templates, but there is also a docker-machine driver that will provision a specific VM with Docker pre-installed.

Digital Ocean has `doctl` and Google GCE has `gcloud`, for example. Both have also a docker-machine driver in place as well. The docker-machine workflow takes care of a lot of things and results in a more portable and consistent process.

Considering the developer convenience and the will to avoid vendor lock-in, it is my belief that the best tool for provisioning small-scale operations is docker-machine. Any reasonably complex project can be provisioned and replicated automatically in a way that the cloud provider becomes a commodity with little impact for development teams.

Cloud providers like Digital Ocean deliver a very simple model of provisioning; others, like AWS or Azure, have a more complex feature set. Still, docker-machine handles all providers gracefully. Remember: less is more.

Sidenotes and finishing up

Using docker-machine, once you get it to work with your cloud provider, is irresistible. You get to control remote Docker engines securely and with little effort; turning these engine into a swarm is also quite easy (and cloud-agnostic).

A few sidenotes…

…on docker-machine

Well, docker-machine is awesomely simple to use — it creates secure nodes and generates the keys for SSL communication between client and engine without a sweat.

Carina CLI does the same, with one practical advantage: it recreates all these client settings anywhere you run it with proper credentials. Unlike Carina CLI, docker-machine settings are kept on the machine that ran the original "docker-machine create" commands.

There are a few "export/import" tools for docker-machine, but those are still a little crude. Just keep that in mind that all you need is in the "$HOME/.docker" folder if you want to rescue it later (or manage the machines from somewhere else).

Notice: I have moved the ".docker" folder from a Mac into a Linux machine and I just had to fix the paths in each "config.json" under "~/.docker/machine/machines".

…on swarm managers

There can be (and should be) several managers in a swarm with high availability. Given the fluid nature of clouds, it is a wise choice to use reserved (fixed) IPs when creating a swarm manager. The "docker-machine" utility has an option for doing this on Azure ("--azure-static-public-ip").

Once again, don't be confused with the "--swarm" options on docker-machine, because they still refer to the old Swarm.

…on proprietary tools and Kubernetes

I have been insisting on calling "CaaS-style" those providers that, at the end of the day, allow you to continue to use the "docker" client naturally. ECS-CLI is not such a case, even though it is somewhat compatible with "docker-compose" syntax.

Until recently Kubernetes was praised as the winning standard for container orchestration (specially if you believe in Gartner reports) — since Docker 1.12 nobody can be that sure. The new "Swarm Mode" is still very crude but surely will evolve very fast, resulting in HA clusters that are managed by the same tools of a single node local development workstation. It is more likely that Swarm Mode will evolve into a complete large-scale solution than Kubernetes will ever become simpler. But, hey, that's just my opinion.

Still Kubernetes is open-sourced and freely used on several cloud vendors (look for "Turn-key Cloud Solutions") — including Google GCE and Amazon AWS, so it is not vendor-locking per se nor proprietary at all. It just forces the developer into a whole new set of tools and overlapping syntax, and I really really really hate that.

…on old swarm and new swarm mode

To keep it bluntly (even if somewhat innacurate):

Old swarm: "docker run", "docker-compose" etc. target the swarm transparently;
Swarm mode (new in Docker 1.12): you use "docker service" instead.

Just be careful if provisioning the cluster with the cloud provider tools — most of them refer to the old swarm (even "docker-machine" does it until version 0.8.1 if you use the "--swarm" option). If you want the new swarm mode be sure to provision simple Docker nodes (with docker-machine or not) and then turn them into a swarm.

…on volumes

Dealing with volumes on swarms is an entire different beast — it makes no sense to rely on local folder mounts for that — containers are expected to "float" around the swarm and local folders kinda kill the whole purpose.

We will cover this subject on the next article.

Building an Ethereum playground with Docker (part 4— Provisioning)

The Point of CaaS and Serverless

Docker on the workstation

Your local tools

Docker on the Cloud

The “setenv.sh” script

Docker Machine (Digital Ocean)

Hey, You Are Working With a Remote Engine

Docker on the Cloud

Your local tools

The “setenv.sh” script

Digital Ocean

Azure Container Service (Azure CLI)

Azure on Docker-Machine

Amazon on ECS-CLI

Amazon on docker-machine

Other Cloud Providers

Sidenotes and finishing up

…on docker-machine

…on swarm managers

…on proprietary tools and Kubernetes

…on old swarm and new swarm mode

…on volumes

Written by André Fernandes