Kubernetes in Turtlemint 🐢

Published in

Turtlemint Engineering Blog

10 min readMar 17, 2020

What is Kubernetes? 🤔

Kubernetes is a modern-day open-source container-orchestration system symbolized as K8s. Designed and developed with an intent to meet different workloads it has revolutionized the cloud computing space. With its rapidly growing ecosystem, it has now become the standard for deploying, managing containerized applications at scale in private, public and hybrid cloud environments.

Developing Complexities Encountered…

Early on, we were deploying each application on a different physical server. However, this system had several drawbacks in the form of underutilized resources and increased expenses on maintaining multiple physical servers. Even today, this continues to be a burning issue as the deployments of the applications have already been done on service/application-specific dedicated servers.

Enterprise customer and cloud-agnostic requirements

For our business needs, it’s required to host the applications on any cloud like AWS, Azure, GCP or on-premises (private data centers). Here, we have to take care of Service Uptime, High Availability, Streamline releases like Automated rollouts and rollbacks, Service Discovery, etc.

Apart from that we also have to look into the Cost Optimisation of the infrastructure & make sure resources are fully used & it should be autoscaled as per need.

While making this automated & cloud-agnostic, this has created a lot of deployment-specific code complexity & duplication in automation jobs like ansible or IaaS tools like terraform. So we required a system that works as a Platform-As-A-Service to orchestrate our application containers.

Kubernetes to your rescue! 🚀

The advent of Kubernetes has rendered multi-facet benefits to the developers. In a production environment, you need to effectively manage the containers that run the applications, ensuring that there is no downtime. Which implies that the shift between the containers has to be smooth. As soon as, the present container goes down, another container needs to supersede. How we wish if this was handled by a system!

That’s how Kubernetes comes to the rescue! Kubernetes not only provides you with a framework to run distributed systems resiliently but also takes care of scaling and failover for your application. Furthermore, it also provides deployment patterns. For example, Kubernetes can easily manage a canary deployment for your system.

Kubernetes carries a number of features that provide a pool of easements.

Container Orchestration

A system for automating deployment, scaling, and management of containerized applications

Cost Optimization

Automatic bin packing: You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.

Self-healing

Takes service uptime to next level (Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve)

Streamlines Releases

Automated rollouts and rollbacks: You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.

Service Discovery

All the container hosts are connected together using an overlay network for providing container-to-container routing. Applications deployed on Kubernetes are dynamically discoverable within the cluster network and can be exposed to the external networks using traditional load balancers. The state of the cluster manager is stored on a highly distributed key/value store etcd which runs within the master instances.

High Availability

Kubernetes scheduler will always make sure that each application component is health checked. It provides high availability when the number of replicas is set to more than one. In that case, each instance is scheduled in multiple hosts, and if one of those hosts becomes unavailable all the containers which were running in that host are scheduled in any of the remaining hosts.

Autoscaling

It provides the ability to auto-scale containers using a resource called Horizontal Pod Auto-scaler which watches the resource consumption and scales the number of containers needed accordingly.

Secret & Configuration Management

Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

Run Anywhere (PaaS)

Will Enables seamless deployments for all SaaS customers

How does it work?

Kubernetes have the potential to provide a set of dynamically scalable hosts for running workloads using containers. The workloads could include long-running services, batch jobs, and container host-specific daemons. It uses a set of management hosts called masters for providing an API to manage the entire container infrastructure.

A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node. The worker node(s) host the pods that are the components of the application. The Control Plane manages the worker nodes and the pods in the cluster.

Components

Control Plane Components (Master)
Node Components

Control Plane Components (Master) :

The Control Plane’s components make global decisions about the cluster (for example, scheduling), as well as detecting and responding to cluster events.

kube-apiserver

The API server is a component of the Kubernetes control plane that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane.

etcd

The consistent and highly-available key-value store used as Kubernetes’ backing store for all cluster data.

kube-scheduler

Control Plane component that watches for newly created pods with no assigned node, and selects a node for them to run on.

Factors taken into account for scheduling decisions include individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, and deadlines.

kube-controller-mananger

Control Plane component that runs controller processes.

These controllers include:

Node Controller: Responsible for noticing and responding when nodes go down.
Replication Controller: Responsible for maintaining the correct number of pods for every replication controller object in the system.
Endpoints Controller: Populates the Endpoints object (that is, joins Services & Pods).
Service Account & Token Controllers: Create default accounts and API access tokens for new namespaces.

cloud-controller-mananger

It runs controllers that interact with the underlying cloud providers.

Node Components :

Node components run on every node, maintaining running pods and providing the Kubernetes runtime environment.

kubelet

An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.

kube-proxy

It is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service (A way to expose an application running on a set of Pods as a network service) concept.

Container Runtime

The container runtime is the software that is responsible for running containers like Docker.

* Kubernetes Objects

Kubernetes contains a number of abstractions that represent the state of your system: deployed containerized applications and workloads, their associated network and disk resources, and other information about what your cluster is doing. These abstractions are represented by objects in the Kubernetes API.

The basic Kubernetes objects include:

Pod
Service
Volume
Namespace

Kubernetes also contains higher-level abstractions that rely on Controllers to build upon the basic objects and provide additional functionality and convenience features. These include:

Deployment
DaemonSet
StatefulSet
ReplicaSet
Job

Pods

A Pod is the basic execution unit of a Kubernetes application–the smallest and simplest unit in the Kubernetes object model that you create or deploy. A Pod represents processes running on your Cluster.

A Pod encapsulates an application’s container, storage resources, a unique network IP, and options that govern how the container should run. A Pod represents a unit of deployment: a single instance of an application in Kubernetes.

Service

An abstract way to expose an application running on a set of Pods as a network service.

Deployments

A Deployment provides declarative updates for Pods and ReplicaSets.You describe the desired state in a Deployment, and the Deployment controller changes the actual state to the desired state at a controlled rate.

How Kubernetes Ingress works

**Kubernetes Ingress with AWS ALB Ingress Controller**

Ingress

An API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination, and name-based virtual hosting.

internet
        |
   [ Ingress ]
   --|-----|--
   [ Services ]

Ingress Creation

Following the steps in the numbered blue circles in the above diagram:

The controller watches for Ingress events from the API server. When it finds Ingress resources that satisfy its requirements, it starts the creation of AWS resources.
An ALB is created for the Ingress resource.
TargetGroups are created for each backend specified in the Ingress resource.
Listeners are created for every port specified as Ingress resource annotation. If no port is specified, sensible defaults (80 or 443) are used.
Rules are created for each path specified in your Ingress resource. This ensures that traffic to a specific path is routed to the correct TargetGroup created.

The Ingress resource routes ingress traffic from the ALB to the Kubernetes cluster.

Create a Kubernetes Cluster

Kubeadm

Kubeadm is a tool built to provide kubeadm init and kubeadm join as best-practice “fast paths” for creating Kubernetes clusters.

Kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines. Likewise, installing various nice-to-have addons, like the Kubernetes Dashboard, monitoring solutions, and cloud-specific addons, is not in scope.

# Install kubectl on ubuntu
$ sudo apt-get install -y kubelet kubeadm kubectl

Kubectl

Kubectl is a command-line tool for controlling Kubernetes clusters. kubectl looks for a file named config in the $HOME/.kube directory. You can specify other kubeconfig files by setting the KUBECONFIG environment variable or by setting the — kubeconfig flag.

# Install kubectl on macos
$ brew install kubectl
$ kubectl version
$ kubectl get pods   # To list pods in cluster

Kops is the easiest medium to get a production-grade Kubernetes cluster up and running.

kops helps you create, destroy, upgrade and maintain production-grade and highly available Kubernetes clusters from the command line. Currently, it supports AWS (Amazon Web Services), GCE, OpenStack, VMware vSphere.
Working with kops

# Install kops on macos 
$ brew install kops
$ kops version# Export cluster name & state store
$ export NAME=staging.k8s.local
$ export KOPS_STATE_STORE=s3://kops.k8s.clusters   # Storing state on S3# Create cluster
$ kops create cluster \
--zones ap-south-1a,ap-south-1b,ap-south-1c \
--master-zones ap-south-1a,ap-south-1b,ap-south-1c \
--networking calico \
--master-size t2.small \
--node-size t3.xlarge# Validate cluster
$ kops validate cluster --name=$NAME# Edit cluster
$ kops edit cluster --name=$NAME# Update cluster
$ kops update cluster --name=$NAME --yes

Working with kubectl for service deployment

#  Create deplyoment.yaml file apiVersion: apps/v1
kind: Deployment
metadata:
  name: motor-service
  namespace: turtlemint
  labels:
    app: motor-service
    env: prod
    service_type: "application-layer"
    cluster_type: "production"
    team: "TM-MOTOR"
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: "25%"
  selector:
    matchLabels:
      app: motor-service
  template:
    metadata:
      labels:
        app: motor-service
        env: prod
        service_type: application-layer
        cluster_type: production
    spec:
      containers:
        - name: motor-service
          image: "prod-motor-service:latest"
          imagePullPolicy: Always
          resources:
            limits:
              memory: "1200Mi"
              cpu: "0.7"
            requests:
              memory: "1000Mi"
              cpu: "0.5"
          ports:
            - containerPort: 9000
          readinessProbe:
            httpGet:
              scheme: HTTP
              path: /api/healthcheck
              port: 9000
            timeoutSeconds: 5
            successThreshold: 2
            failureThreshold: 3
            periodSeconds: 2
            initialDelaySeconds: 5
          envFrom:
            - configMapRef:
                name: "motor-service-configmap"

#  Create configmap.yaml file to set environment variablesapiVersion: v1
kind: ConfigMap
metadata:
  name: "motor-service-configmap"
  namespace: "turtlemint"
data:
 NEW_RELIC_LICENSE_KEY: "xxxxxxxxxxxxxxxxxx"
 NEW_RELIC_APP_NAME: "PROD_MOTOR_SERVICE"
 NEW_RELIC_DISTRIBUTED_TRACING_ENABLED: "true"

# Create service.yaml file for defines a logical set of Pods and a policy by which to access them (sometimes this pattern is called a micro-service)apiVersion: v1
kind: Service
metadata:
  name: motor-service
  namespace: turtlemint
spec:
  ports:
    - port: 8080
      targetPort: 8080
      protocol: TCP
  type: NodePort
  selector:
    app: motor-service# With Kubernetes you don’t need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.

# HPA (Horizontal Pod Autoscaler) 
# The Horizontal Pod Autoscaler automatically scales the number of pods in a deployment based on observed CPU utilization (or, with custom metrics support).# Create hpa.yaml file for autoscaling policies of serviceapiVersion: autoscaling/v2beta1      
kind: HorizontalPodAutoscaler
metadata:
  name: motor-service
  namespace: turtlemint
spec:
  maxReplicas: 10
  minReplicas: 4
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: motor-service
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 80
    - type: Resource
      resource:
        name: memory
        targetAverageUtilization: 80

# Deploying the service
$ kubectl apply -f deployment.yaml# Watch on service pods
$ kubectl get pods --selector=app=motor-service -w# Describe the service pod
$ kubectl describe pod motor-service

Manage Kubernetes Applications

If you need help in managing Kubernetes applications, Helm would serve as a desired Kubernetes package manager. Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.

Conclusion ✍️

All in all, we can rightly conclude that Kubernetes has not only dramatically reduced the cost of cloud computing but also made it easy and hassle-free. It has also played a crucial role in simplifying new-age system architectures. Thus, rendering support to organizations in overcoming their day to day development and operational challenges.