Docker Swarm at AWS/Azure vs. Amazon EC2 Container Service vs. Microsoft Azure Container Service (Swarm)

The purpose of this article is to take a quick look at some top features comparison in Docker containers orchestration solutions for two major Cloud Providers. As the title states, we will take Microsoft Azure Container Service (Swarm) vs. Amazon EC2 Container Service. Then we will look at Docker Swarm Mode built-in orchestration functionality that works almost identically independently of host provider just to show what to expect if you don’t want to deal with “service jail”.

This should leave you with a question:
“Is built-in Docker Container Orchestration on Cloud Providers worth it?”

I know, that it is not completely fair to compare Microsoft Azure Container Service (Swarm) with Amazon EC2 Container Service.
However, after evaluating other choices like Kubernetes and DC/OS, which Azure currently adopts support of, I figured out that the most common orchestration competitor will be the Docker Swarm.

First we can compare functional capabilities of container orchestration tools using matrix tables provided below.
Then functional capabilities and non-functional qualities details will be described in short form.
To avoid any holy-wars or controversies all functional features go with a brief description of meaning related to this particular article.

If orchestration tool in the very basic scenario does the job mentioned in the description, then it is considered as supported. If the job could be covered by other service or component of Cloud Provider, or it could be done by little reconfiguration, then it is considered as partially supported. If the job requires full reconfiguration of container cluster or just can’t be done using public version of the Docker Engine, then it is considered as not supported.

Don’t expect an advanced comparison of functional capabilities and especially non-functional qualities, because it would take an enormous amount of time and would look like a small version of Wikipedia…

Legend of icons used in comparison matrix tables:

Supported
Not supported
Partially supported

- Feature is supported in a matter of orchestration

- Feature is unsupported in a matter of orchestration

- Feature support is covered by other service or component of Cloud Provider


Microsoft Azure Container Service (Swarm)

This is the Docker Engine you will get in the end of deploying an Azure Container Service cluster using Swarm template or going through ACS deployment wizard using Azure Portal:

Client:
Version: 1.12.3
API version: 1.24
Go version: go1.6.3
Git commit: 6b644ec
Built: Wed Oct 26 21:44:32 2016
OS/Arch: linux/amd64
Server:
Version: swarm/1.1.0
API version: 1.22
Go version: go1.5.3
Git commit: a0fd82b
Built: Thu Feb 4 08:55:18 UTC 2016
OS/Arch: linux/amd64
If you want the so called “swarm mode” introduced in the Docker Engine 1.12, then you should change a lot in configure-swarm-cluster.sh, reconfigure ports and networks or just configure everything manually without using ACS at all. But this is another story.

FUNCTIONAL CAPABILITIES:

Scheduling

  • Placement
    Choose where to run the container: Supported
  • Replication
    Mirroring of container information to other instances: Partially — only for Managers (if the primary manager fails, a secondary can become the primary manager)
  • Scaling
    Scale up or down the number of container instances: Supported (only manually starting and killing one by one)
  • Auto Scaling
    Automatically scale up or down the number of container instances: Not supported
  • Readiness Checking
    Container-specific way to determine readiness, health checking: Supported
  • Resurrection
    Restart the container if it stops on the same node: Supported
  • Rescheduling
    Restart the container if it stops on another node: Not supported
  • Rolling Deployment
    Container upgrades without downtime: Not supported
  • Upgrades/Downgrades
    Docker engine in-place upgrades/downgrades: Not supported
  • Collocation
    Run multiple containers that have relations between them on the same node: Not supported

Resource management

  • Memory
    Ability to read configuration and use memory from hosts: Supported
  • CPU
    Ability to read configuration and use CPU from hosts: Supported
  • GPU
    Ability to read configuration and use GPU from hosts: Not supported
  • Disk Space
    Ability to read configuration and use Disk Space information from hosts: Not supported
  • Volumes
    Ability to read configuration and store data depended of host/container configuration: Supported
  • Persistent Volumes
    Ability to read configuration and store data independently from the host/container configuration: Partially — by logging into each agent and configuring the Docker Engine on that host
  • Ports
    Port mapping ability: Supported
  • Network
    Ability to create and manage internal/overlay networks: Partially — only by changing predefined subnets for masters and agents in template prior to deploying or going through ACS deployment wizard
  • Scaling
    Scale up or down the number of hosts: Partially — by resubmitting the template with a new value for the number of agents
  • Auto scaling
    The creation or deletion of hosts as needed to match performance requirements: Partially — by using VM Scale Set

Service Management

  • Labels
    The key/value pairs that allow to distinguish containers and/or tags hosts: Supported
  • Groups/Namespaces
    Virtual groups or clusters within global cluster: Not supported
  • Dependencies
    Bonds between containers like order of start, links between, shared resources: Not supported
  • Load Balancing
    Distribute load across container instances: Partially — using Azure Application LB. You need to change template prior to deployment, or change it using Portal if you need custom ports. Default ports are: 80, 443, 8080
  • Service Checking
    The way to tell if the service inside container is up and available: Not supported

NON-FUNCTIONAL QUALITIES:

Scalability

Availability

Flexibility

  • Format Support — Linux based Docker containers only
  • Portability — Simply reprodusable on other Cloud Providers or on Premices
  • Interoperability — Docker legacy standalone Swarm only
  • Extensibility — No

Usability

  • Familiarity — Shares the same principles and resources as all Azure Services
  • Maintainability — No
  • Compatibility — All tools for managing containers on a Swarm cluster using Docker API are supported
  • Debuggability — No

Portability

  • Container Runtimes — Docker Engine
  • Host OS — Ubuntu Server 14.04.4-LTS
  • Hosted — Not possible, but reprodusable
  • Cloud — Microsoft Azure
  • Bare-metal — Not possible

Security

  • Auditability — 3rd Party Utilities only
  • Secrets Management — 3rd Party Utilities only
  • Encryption — Not possible withing cluster
  • Isolation — Supported, can be deployed in private VNet only

Amazon EC2 Container Service

This is the Docker Engine you will get in the end of deploying an Amazon Container Service cluster using AWS CloudFormation template or going through ECS deployment wizard using AWS Management Console:

Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.3
Git commit: b9f10c9/1.11.2
Built:
OS/Arch: linux/amd64

As you can see, I didn’t provide details of Master/Manager Docker Engine, because there is no any. ECS has it is own container management service that supports Docker containers and provides single source of coordinating the cluster.

FUNCTIONAL CAPABILITIES:

Scheduling

  • Placement
    Choose where to run the container: Supported (only by using AWS ECS CLI)
  • Replication
    Mirroring of container information to other instances: Not supported
  • Scaling
    Scale up or down the number of container instances: Supported
  • Auto Scaling
    Automatically scale up or down the number of container instances: Partially — by using Amazon CloudWatch
  • Readiness Checking
    Container-specific way to determine readiness, health checking: Supported
  • Resurrection
    Restart the container if it stops on the same node: Not supported
  • Rescheduling
    Restart the container if it stops on another node: Supported
  • Rolling Deployment
    Container upgrades without downtime: Supported
  • Upgrades/Downgrades
    Docker engine in-place upgrades/downgrades: Partially — by using Amazon EC2 Systems Manager
  • Collocation
    Run multiple containers that has relations between them on the same node: Supported

Resource management

  • Memory
    Ability to read configuration and use memory from hosts: Supported
  • CPU
    Ability to read configuration and use CPU from hosts: Supported
  • GPU
    Ability to read configuration and use GPU from hosts: Not supported
  • Disk Space
    Ability to read configuration and use Disk Space information from hosts: Not supported
  • Volumes
    Ability to read configuration and store data depended of host/container configuration: Supported
  • Persistent Volumes
    Ability to read configuration and store data independent of host/container configuration: Supported
  • Ports
    Port mapping ability: Supported
  • Network
    Ability to create and manage internal/overlay networks: Partially — by using Amazon Virtual Private Cloud prior to deploying or going through ECS deployment wizard
  • Scaling
    Scale up or down the number of hosts: Supported
  • Auto scaling
    The creation or deletion of hosts as needed to match performance requirements: Partially — by using EC2 AutoScaling Group

Service Management

  • Labels
    The key/value pairs that allow to distinguish containers and/or tags hosts: Supported
  • Groups/Namespaces
    Virtual groups or clusters within global cluster: Supported
  • Dependencies
    Bonds between containers like order of start, links between, shared resources: Partially — by using AWS CloudFormation
  • Load Balancing
    Distribute load across container instances: Partially — using AWS LB
  • Service Checking
    The way to tell if the service inside container is up and available: Not supported

NON-FUNCTIONAL QUALITIES:

Scalability

  • Performance — No matter how large the cluster size is, but has some limitations
  • Responsiveness — There are no guarantees about performed operations
  • Efficiency — There is no support of hosts compute ranking built-in

Availability

Flexibility

Usability

  • Familiarity — Shares the same principles and resources as all Amazon Web Services
  • Maintainability — No
  • Compatibility — Amazon EC2 Container Service WebUI/CLI or 3rd Party Utilities
  • Debuggability — No

Portability

  • Container Runtimes — Docker Engine
  • Host OS — Any Linux Based OS with Amazon ECS Container Agent installed. Windows OS Support currently in Beta
  • Hosted — Not possible
  • Cloud — Amazon Web Services
  • Bare-metal — Not possible

Security

  • Auditability — AWS CloudTrail
  • Secrets Management — Amazon Identity and Access Management + Amazon Simple Storage Service
  • Encryption — Not possible withing cluster
  • Isolation — Supported

Docker Swarm Mode at AWS or Azure

This is the Docker Engine you will get in the end of installing likecurl -fsSL https://get.docker.com/ | sh on Amazon EC2 or on Azure VM:

Client:
Version: 1.12.3
API version: 1.24
Go version: go1.6.3
Git commit: 6b644ec
Built:
OS/Arch: linux/amd64
Server:
Version: 1.12.3
API version: 1.24
Go version: go1.6.3
Git commit: 6b644ec
Built:
OS/Arch: linux/amd64

As you can see, I didn’t provide details of Master/Manager Docker Engine, because this is a built-in functionality. If you’re creating the cluster on host, it is automatically joining as Master/Manager node.

FUNCTIONAL CAPABILITIES:

Scheduling

  • Placement
    Choose where to run container: Supported
  • Replication
    Mirroring of container information to another instances: Supported
  • Scaling
    Scale up or down the number of container instances: Supported
  • Auto Scaling
    Automatically scale up or down the number of container instances: Not supported
  • Readiness Checking
    Container-specific way to determine readiness, health checking: Supported
  • Resurrection
    Restart the container if it stops on the same node: Supported
  • Rescheduling
    Restart the container if it stops on another node: Supported
  • Rolling Deployment
    Container upgrades without downtime: Supported
  • Upgrades/Downgrades
    Docker engine in-place upgrades/downgrades: Not supported
  • Collocation
    Run multiple containers that has relations between them on the same node: Supported

Resource management

  • Memory
    Ability to read configuration and use memory from hosts: Supported
  • CPU
    Ability to read configuration and use CPU from hosts: Supported
  • GPU
    Ability to read configuration and use GPU from hosts: Not supported
  • Disk Space
    Ability to read configuration and use Disk Space information from hosts: Not supported
  • Volumes
    Ability to read configuration and store data depended of host/container configuration: Supported
  • Persistent Volumes
    Ability to read configuration and store data independent of host/container configuration: Supported
  • Ports
    Port mapping ability: Supported
  • Network
    Ability to create and manage internal/overlay networks: Supported
  • Scaling
    Scale up or down the number of hosts: Not supported
  • Auto scaling
    The creation or deletion of hosts as needed to match performance requirements: Not supported

Service Management

  • Labels
    The key/value pairs that allow to distinguish containers and/or tags hosts: Supported
  • Groups/Namespaces
    Virtual groups or clusters within global cluster: Supported
  • Dependencies
    Bonds between containers like order of start, links between, shared resources: Supported
  • Load Balancing
    Distribute load across container instances: Supported
  • Service Checking
    The way to tell if the service inside container is up and available: Not supported

NON-FUNCTIONAL QUALITIES:

Scalability

Availability

Flexibility

  • Format Support — Linux based Docker containers. Windows OS based containers support currently lacks some features
  • Portability — Other Cloud Providers, Virtual Machines, Supported OSs
  • Interoperability — No
  • Extensibility — Any product that supports Docker Swarm Mode API

Usability

  • Familiarity — Shares the same principles and resources as all Docker products
  • Maintainability — Change node availability
  • Compatibility — Docker Engine v. 1.12+
  • Debuggability — Daemon debugging

Portability

  • Container Runtimes — Docker Engine
  • Host OS — Any Linux Based OS with Docker service installed. Windows OS Support currently lacks some features
  • Hosted — Supported
  • Cloud — Amazon Web Services/Azure/Any other that provides Virtual Machines
  • Bare-metal — Not possible

Security

  • Auditability — 3rd Party Utilities only
  • Secrets Management — No
  • Encryption — TLS
  • Isolation — Supported