An Overall View On Docker Ecosystem — Containers, Moby, Swarm, Linuxkit, containerd, Kubernetes ..
The goal of this blog post (and the video) is sharing an an overall view of containers technologies. We’re not going through many technical details, instead of that we’re going to have a global view on containers and Docker.
We’ve seen a lot of changes in Docker since its first version and this could be confusing for engineers and developers trying to learn this technology.
That’s why we’re going to see different concepts from the containers ecosystem, the relationship between them, an introduction to Docker as well as its most important milestones until 2018.
The Complete Video:
This video was created to be included in Painless Docker guide. You can watch it here and buy the guide here. If you liked the video, you can share it and subscribe to the channel.
All the images are taken from docker.com website.
Container as a technology is not new but the popularity of Docker could make some people think that it’s the only container technology. In reality, there are many others. Let’s enumerate some of them.
The following list is ordered from the least to the most recent technology:
- Chroot Jail
- FreeBSD Jails
- Solaris Containers
- Process Containers
It was introduced in 1979 and it’s considered as one of the first containerization technologies It allows you to isolate a process and its children from the rest of the operating system
The FreeBSD Jail
The freebsd jail has an implementation of OS-level virtualization. It was one of the first virtualization technologies at the OS level
A virtual private server using OS-level virtualization capabilities that was added to Linux kernel
Oracle Solaris Containers
It’s also an OS level virtualization technology and it’s designed for X86 and SPARC system
A solaris containers is a combination of system resource controls and the boundary separation provided by “zones”
Openvz, an OS level virtualization technology for Linux. It allows you to create multiple secure isolated Linux containers known as VPS
Process containers was developed by engineers from Google. It’s more known as cgroups or Control Groups.
(Linux containers, or LXC)
It’s an OS level virtualization technology that allows running multiple isolated systems on a control host using a single Linux kernel.
At its initial stage, it used LXC as a container runtime. It was later replaced with a CloudFoundy implementation.
(Let me contain that for you)
It’s the Open source version of Google’s container stack.
Google engineers have been collaborating with Docker team over libcontainer and porting the core concepts and abstractions to this project.
The project is not actively being developed but in the future the core of this project will be probably replaced by libcontainer.
Docker is the tool that can package an application and its dependencies in a container that can run almost on any server
It’s an application container engine focused on security and open standards.
As we have seen , Docker is not the first containerization technology but it’s the most know one.
This technology was introduced in 2013 and was changing and evolving over the last years.
These are the main components of Docker platform.
Docker stands between the infrastructure and the application stack and
It’s composed of an industry-standard container runtime called containerd, a native orchestration tool called docker Swarm, Docker Community edition which is the Open Source version of Docker And the enterprise edition that provides commercial management service.
Lets dive deeper into some concepts and tools like containerd and LXC
Docker & LXC
The first execution environment of Docker was LXC but it was later replaced by libcontainer starting from the version 0.9
Docker & libcontainer
Libcontainer is a Docker interface to Linux facilities like Cgroups, namespaces, netlink and netfilter.
2015 — Docker & runC
- In 2015, Docker announced runC: a lightweight, portable container runtime. It’s basically a little command-line tool to leverage libcontainer directly, without going through the Docker Engine
The goal of runC is to make standard containers available everywhere. This project was donated the Open Container Initiative.
Docker & The Open Containers Initiative
The OCI is a lightweight, open governance structure that was launched on 2015 by Docker, CoreOS and other leaders in the container industry.
It maintains some projects like runC, the runtime and the image specifications.
Its purpose is developing standards around the container industry so if you create a container using Docker for example, you can run on any other engine.
2016 — Docker & containerd
In 2016, Docker spinned out containerd and donate it to a new community project.
Breaking out this component into a separate project allows Docker to move the container supervision out of the core Docker engine and into a separate daemon.
So, Docker moved from a monolithic software to a set of independent components and projects .
How Docker Runs a Container ?
- Docker engine creates the image,
- pass it to containerd,
- containerd calls containerd-shim,
- containerd-shim uses runC to run the container,
- containerd-shim allows the runtime (runC in this case) to exit after it starts the container
The main two benefits of this model are
- Running deamon less containers
- The ability to restart or upgrade engine without breaking the running containers
2017 — Containers Goes Mainstream
2017, was the year during which containers went mainstream that’s why Docker was building several Docker editions beyond Linux (Docker for Mac, Docker for Windows, Docker for AWS, GCP ..etc).
With this container’s mass adoption, Docker inc realized that new production models were needed and that’s why it started the Moby project.
The Moby Project
The Moby project was started to enable a new level of collaboration and production. It’s an open source project with the aim of advancing the software containerization movement
It provides a lego set of dozens of components and the framework for assembling them into custom container-based systems.
Docker production model started like any other common open source monolithic project
It moved to splitting out a single project to different open components
Then to a model that allows sharing these components and assemblies
And finally to a model that provides more collaboration on components and commons assemblies
Let’s now see some components of the Moby project
Containerd is the industry based core container runtime for Docker.
It’s available as a daemon for Linux and windows and it manages the complete container lifecycle like the image transfer and storage, container execution and supervision, low level storage and the network attachments..
Linuxkit, another component of the Moby project and it’s a tool to build secure, portable and lean operating systems for containers.
It’s currently supported by
- local hypervisors like hyper-v and vmware.
- Some cloud based platforms like AWS, GCP and Azure
- And baremetal on packet.net
Infrakit is also part of the Moby project.
It’s a toolkit for creating and managing declarative, immutable and self healing infrastructures.
Infrakit is designed to automate setup and management of infrastructure in support of distributed systems and higher level container orchestration systems.
Infrakit is useful for some use cases like bootstrapping orchestration tools like Docker Swarm and Kubernetes or creating autoscaling clusters across some public cloud like AWS and it’s autoscaling groups.
Libnetwork is a native Go implementation for connecting containers.
It supports the development of network drivers and plugins and it aims to satisfy the “composable” need for networking in containers.
Docker & Docker Swarm
It’s an orchestration tool built in Docker engine.
It started as a standalone tool and was natively included in Docker starting from the version 1.12.
It uses the Docker CLI to create a swarm cluster, deploy and manage applications and services.
Docker & Kubernetes
One of the most important changes is integrating Kubernetes natively into Docker . This was announced In October 2017
With this integration, Docker customers and developers have the option to use both Kubernetes and Swarm to orchestrate container workloads.
The coming versions of Docker with Kubernetes support will allow users to deploy their Docker Compose apps as Kubernetes-native Pods and Services.
Kubernetes is considered as a native orchestration tool just like Swarm.
Docker Pulls From 2013 To 2017
In this blog post we’ve seen the different milestones and the evolution of Docker, and discovered some tools like libcontainer, libnetwork, RunC, Swarm, Containerd and Linuxkit.
Don’t forget to check PainlessDocker.com to get more details about this guide.
You can also test your Docker knowledge using this quiz that I created for the my readers !
If you are interested in joining my community DevOpsLinks, you can apply here, you’l receive a weekly newsletter and some other occasional emails including must-read articles, news and tutorials.
You can show your support by liking and sharing the video and/or giving this article a big clap :-)