Kubernetes and Other Orchestrators

Hi! My name is Leonid, I am a DevOps engineer at KTS.

In this article, I will review different orchestrators and explain why Kubernetes is the best choice.

Our company is 6 years old, and we’ve been living with Kubernetes for 4 of them. Before that, we tried all kinds of app deploy options on the servers: starting with a simple git pull up till ci/cd on multiple servers.

In the short period of four years we have gained a lot of experience. We started with the question you may be facing right now: “Which orchestrator to choose?” We considered different options, and in the end we settled on Kubernetes. In this article I will tell you why.

What’s in the article:

• What is an orchestrator and what tasks it performs
• Docker Swarm
• Architecture
• Pros
• Cons
• Nomad
• Architecture
• Pros
• Cons
• Kubernetes and OpenShift
• Architecture
• Pros
• Cons
• How to start using
• Conclusion: our case

What is an orchestrator and what tasks it performs

Many of you must have heard of the microservice architecture.

Most microservices are containers that have to work together cohesively. In addition to containers, these can be the usual apps: for example, in Java or on a virtual machine.

The orchestrator allows you to centralize the management:

What a good and handy orchestrator should have:

• The ability to get started quickly, so you don’t have to spend half a year studying and testing.
• A set of certain features right out of the box:
  — Managing secrets: the ability to pass tokens, passwords, and database accesses to the app, rather than storing everything in code.
  — Service discovery: the ability to connect new app replicas to the infrastructure.
  — Cluster customization: the ability to add our entities and organize APIs for our purposes.
• Least possible vendor lock-in. When developers bind the buyer to a particular technology, switching vendors becomes very difficult.

Horizontal scaling: automatic scaling of cluster nodes and the app. The workload can be non-linear, so the orchestrator should be able to increase the number of app replicas on the go. If the cluster lacks resources, you should be able to increase.

Vertical scaling:the cluster manages the resources that are allocated to the app. Ability to allocate and limit resources is necessary to avoid situations when one of the apps has consumed all the memory and CPU on our servers, and others are left with nothing. The orchestrator understands the initial resource limits for the app, and how much they can grow.

• Different strategies for updating the app: the apps are constantly updated. Orchestrators have different strategies to ensure that this goes unnoticed by the user and the business, and that the updates themselves don’t crash in the process:
  – Rolling: a new version of the app is rolled out gradually. For example, we have 5 replicas of an app that gradually replace the original one, until finally the new version takes the place completely.
  – Recreate: a fairly simple strategy where we just kill the old version and roll out the new one.
  – Blue/Green: we roll out two versions of an app at once: while one works, the second is being tested and will eventually replace the first.
  – Canary (Dark): quite similar to Blue/Green, but with one peculiarity. The new version is rolled out to a limited number of users first. If there are no problems, we will gradually replace the entire app with the new version.

Infrastructure as Code (IaC) with templating mechanisms: is a handy thing that lets you store app states, that are described by the code, in some manifests. It is convenient for both the engineers and the newcomers. When a company hires a newcomer, it’s easy for them to get familiarized with the infrastructure and understand how things are set up. And everything is conveniently versioned. We need templating to reuse manifests and not to write the same thing again and again. For example, we substitute different values in prepared templates and change the versions of the app:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.name }}
  namespace: {{ .Values.namespace }}
  labels:
    app: {{ .Values.name }}
spec:
  replicas: {{ .Values.replicas }}
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: {{ .Values.name }}
  template:
    metadata:
      labels:
        app: {{ .Values.name }}
    spec:
      containers:
      - name: {{ .Values.name }}
        image: {{ .Values.image }}
        ports:
          - containerPort: {{ .Values.HTTP_PORT }}

• A large community is the lifeblood of technology and tool diversity. As long as there is a community, the technology will live on.

Docker Swarm — orchestrator built into Docker

Architecture and description:

Docker Swarm consists of managers and workers. Managers determine on which workers and how the containers will run. They also store the state of the cluster.

The Docker Swarm’s minimum working entity is a Service, which is broken down into tasks. Each task has a Docker container:

In order to run some kind of Service, we need to:

1. Run a YAML file, which is very similar to Docker Compose. The only difference is a few parameters specific to Swarm. For example, replicas.
2. Run a command that is very similar to the command for Docker Compose.

version: "3"
 
services:
  proxy:
    image: nginx:latest
    ports:
      - 80:80
      - 443:443
    networks:
      - proxy
    deploy:
      replicas: 1
  
  frontend:
    image: kts/my-super-web:latest
    networks:
      - proxy
    volumes:
      - /opt/images:/usr/share/nginx/images
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 3
   
networks:
  proxy:
    external: true

$ docker stack deploy --compose-file docker-compose.yml stackdemo

Pros

• Completely free
• Not demanding resource-wise

Has Roling Back — the ability to roll back the app to the previous version, if problems arise.

• Low entry level: simple initial setup. You only need to run two commands:

1. To initialize а cluster:

$ docker swarm init --advertise-addr 192.168.99.100

2. To add virtual machines to the clusters:

$ docker swarm join --token %sometoken% 192.168.99.100:2377

Cons

• Only supports Docker containers
• Does not support automatic scaling at all. You will have to create the number of app replicas manually in the replicas parameter:

$ docker service create --name my_web \
                        --replicas 3 \
                        --publish published=8080,target=80 \
                        nginx

• Supports the Rolling Update strategy only.
• Not much support from the community. Many have heard the phrase “Docker Swarm is dead.” The technology is not popular among specialists, it has few ready-made solutions.
• We write the manifest language in YAML, but Docker Swarm has no built-in support for templates.

Nomad — orchestrator by HashiCorp

Quite similar to Docker Swarm:

It consists of the servers that determine where and how the orchestration objects live, and the clients on which those objects run.

Unlike Services in Docker Swarm, in Nomad we create job entities. In their turn, they consist of task entities.

To create a job, we write a manifest and pass it to the server, which then distributes it to the clients.

$ nomad job run example.nomad

job "example-job" {
  type = "service"
  update {
    stagger = "30s"
    max_parallel = 1
  }
  group "example-group" {
    count = 3
    task "example-task" {
      driver = "docker"
      config {
        image = "nginx:latest"
        
        port_map {
          http = 8080
        }
      }
      service {
        port = "http"
        check {
          type = "http"
          path = "/"
          interval = "10s"
          timeout = "2s"
        }
      }
      resources {
        cpu = 500
        memory = 128
        
        network {
          mbit = 100
          port "http" {}
        }
      }
    }
  }
}

Nomad fully supports the entire HashiCorp ecosystem:

• Vault for storing secret passwords, tokens, and more
• Consul for auto-discovery and dns
• Terraform for configuration management

Another feature of Nomad is that it can manage GPU resources of your server, so you can use it for machine learning projects.

It can also coexist with Kubernetes. You may even run some of its jobs directly in Kubernetes.

It has an Enterprise version with some paid features. For example, scaling has just appeared and is implemented through plugins. Both types of horizontal scaling are supported, but vertical scaling is only available in Enterprise version:

Pros

• One of the main differences between Nomad and other orchestrators is that it can orchestrate more than just Docker containers. And generally — not containers only. This can be virtual machines, Java binaries, or even Amazon Elastic Container Service.
• Works on all popular operating systems.
• Not demanding resource-wise: about the same as Docker Swarm.
• Low entry level: simple initial setup. To run it, you need to:
  — Install the Nomad package
  — Structure the servers: you basically only need to specify the addresses of the machines on which it will run
  — Start Nomad
• Describing the job, in the driver = “docker” string we start the Docker container and describe some config related to Docker. The beauty of Nomad is that since it can run more than just Docker containers, the job description will not change much. For example, you run some job as systemd.service, or you have a Java app. You describe the job and run it on your server. If at some point you decide to switch to Docker, you will not need to globally rewrite the job. Just replace the driver and change the deploy structure a bit, but the manifest description process will stay the same.
• It supports all of the popular and necessary upgrade strategies described at the beginning of this article.

Cons

• Few features out of the box:

• There is no load balancing, no autoscaling, no option to add custom controllers.
• The support of the community is a bit better than the Swarm has, but still not much.
• Manifestos are written in the HCL language, which is not common among engineers. If you want to use Nomad, you will have to familiarize yourself with this language:

job "{{service_name}}" {
  datacenters = [{{list_of_datacenters}}]
  
  type = "service"
  
  group "{{service_name}}" {
    count = {{count}}
    }
  task "{{service_name}}" {
    driver = "docker"
    config {
      image = "{{docker_image}}:latest"
    port_map = {
      http = {{http_port}}
    }
  }
  
  service {
    name = "{{service_name}}"
    port = "http"
    }
  }
}

Kubernetes and OpenShift

Kubernetes is an open source project for orchestrating containerized apps.

OpenShift is a family of containerization software products developed by Red Hat.

Architecture and description

We review these orchestrators together because OpenShift has Kubernetes “under the hood.”

Kubernetes is closer to the concepts of framework and technology.

OpenShift is more of a ready-to-use platform.

Kubernetes was born within the walls of Google and then transferred to open source. Many large companies use Kubernetes. It works on Linux and Windows.

OpenShift runs on Red Hat Enterprise Linux exclusively. It only has an Enterprise version with a short trial period. In addition to Kubernetes, it has many other technologies: for code delivery, monitoring, logging, etc.

There are many ways to install Kubernetes:

• manually, installing Kubernetes binaries on the servers, and dealing with certificate issuance and updates yourself
• passing this on to relatively automated tools, such as kubespray and kubeadm
• use managed solutions

The ways to install OpenShift also vary greatly, depending on where and how you deploy it.

Both have utilities letting you try them out. With any of them, you can deploy a full-fledged cluster on your machine and see what Kubernetes or OpenShift are all about:

Kubernetes: minikube, kind, k3s

$ minikube start --vm-driver-docker

$ kind create cluster

$ k3s server &

OpenShift: minishift

$ minishift start

Both orchestrators have a complex architecture.

In Kubernetes, a cluster consists of standard nodes and a Control Plane, which is a master node.

Control Plane includes:

• kube-api-server, that is the central entry point for all Kubernetes components and people who work with the cluster. It provides an API for cluster management.
• kube-controller-manager manages controllers, such as the node controller, which control a node, or a replica controller, which controls the number of pods deployed in our cluster.
• kube-sheduler determines when and on which node the pod will run.
• etcd repository stores the cluster states in key-value format.
• cloud-contoller is an optional feature. It monitors the controllers of a particular public or private cloud.

What runs on the nodes:

• kubelet, which deploys and monitors containers.
• kube-proxy, which handles the network: it configures the network and network rules so that pods on different nodes could communicate with each other.

Kubernetes provides a wide range of scalability options.

With vertical scaling, we can define the request, the amount of resources that will be given to the app at startup, and the limits until which it can expand in digesting resources.

Tracking the average workload, Kubernetes can monitor replicas and nodes, adding or reducing their number. It works the same way in the cloud, buying or selling nodes:

The minimum working entity of both orchestrators is a pod: a container that can be based on Docker, cri-o, and containerd. But, unlike in Nomad which can orchestrate virtual machines and many other things, it will necessarily be a container.

There doesn’t have to be only one container in the pod though. There can be several of them, and they will share the network and file system between each other.

To create a pod, we write a simple YAML file and either apply it or run it directly from the command line. To manage Kubernetes, we use the rather popular kubectl utility.

Using operators, we can create our entities:
PostgreSQL
RabbitMQ

For Postgres, we describe all the necessary parameters. For example, the size of the cluster, the storage, and the databases that will be created for users.

The table below shows why Kubernetes is the industry standard:

Pros

• Easy to run in public clouds.
• Out of the box, Kubernetes supports only two update strategies: Rolling and Recreate. But with extensions you can easily add both Blue/Green and Canary.
• Supports Rollback: monitors pod status and health-check. These are commands that can be configured to help Kubernetes better understand what’s going on with the app. In case of problems, it will also roll back to the previous version.
• Kubernetes allows you to describe your manifests in YAML and JSON. YAML is preferred by the majority of users.
• With the Helm batch manager you have a fairly advanced tool for upgrading manifests. You can use loops, string editing, etc.
• Kubernetes has a large number of built-in features. It has everything you need from an orchestrator, and you can constantly expand its capabilities with custom APIs, network plugins, cluster installation and management tools, etc:

• Kubernetes has a developed community that is very interested in the technology. You can solve almost any problem, or find an engineer who works with Kubernetes.
• The simultaneous pro and con of OpenShift is that the platform is pre-defined for you. On the one hand, you can just take and use it, but, on the other hand, it is not a sure thing that this platform is the best choice for you. You may have to put some other solution side by side with OpenShift and use something here and something there. Therefore, there is a vendor lock-in here: moving from OpenShift to some other orchestrator will be quite a problem.
• Since it is Enterprise, OpenShift has commercial support. Kubernetes does not have it, but it has a large open source community, detailed documentation, and lots of articles and books. Plus, a huge choice of technologies for the user.
• OpenShift offers built-in support for templates.

Con

Both are quite difficult to install on your own.

How to start using Kubernetes

There are a few options to get you started:

• Obviously, you just have to start using it.
• Try managed versions from different cloud providers.
• Contact companies that specialize in operating k8s.
• Take specialized learning courses.

Conclusion: our case

Obviously, the entry level for Kubernetes or OpenShift is higher than in other solutions. But in the end it is a winning line. When we were choosing an orchestrator, in the end we settled on two options: Kubernetes or Nomad.

The shortcomings of Docker Swarm were obvious, and it was clear that in the long run it would definitely not work for us. We deploy a large number of apps that are not connected to each other. We need dev, stage, and prod environments, and it’s hard to do with Docker Swarm.

But we had an existing architecture that didn’t use any of the orchestrators. We used SaltStack to deploy apps to our servers, i.e. we installed them as packages.

We thought about the easiest way to switch to a different technology. Nomad was very attractive in this respect because it can run certain tasks on the machine. We thought, “Now we’re going to switch the deployment to Nomad, then maybe we’ll start using Docker containers, and then will find some other workaround”.

But after thinking about it some more, we came to the conclusion that Nomad has a lot of moving parts and a small community. The main reasons why we chose the k8s were:

1. A big community.
2. Better is included: Kubernetes already has everything you need, and for the most part, you don’t have to look for anything. Of course, you need to get lots of things done in each cluster: to work with TLS certificates, to have a more transparent and convenient deploying, maybe a service mesh. All these things are not built in, but there is a large number of ready solutions.

Here we should mention the CNCF organization, which promotes container solutions and has an incubator to develop them. One such solution was Kubernetes itself: it was the first project that the CNCF released. After that there were solutions for managing ceph clusters, Ingress Controller, and many more. And it’s very important that there is a tremendous support from the community and the nonprofit organization that manages the development of container technology. There are a lot of companies that contribute to both CNCF and Kubernetes. That’s why we decided it was the best choice.

The transition was not easy. We had to write a lot of things from scratch and to transfer all the applications into Docker. In about six months we moved all the projects to the new technology. We have been living in Kubernetes ever since. Of course, we have separate servers associated with databases that are deployed separately, but the apps themselves live almost entirely in Kubernetes.

Share a comment: what orchestrators do you work with? Are there pros and cons that we forgot to mention?