Achieving Resilience: High Availability Strategies in Kubernetes

Introduction

In cloud computing, it’s important to keep services running smoothly, even when maintenance tasks like updating or restarting nodes are necessary. Kubernetes (K8s) is widely used to manage, scale, and deploy applications. However, keeping services available during node updates can be challenging.

Cloud providers often need to restart or stop cluster nodes for maintenance, updates, or security reasons. Without proper strategies, these actions can disrupt services and affect user experience.

To prevent interruptions, it’s essential to use Kubernetes features and best practices. This includes setting up

1. Multi replica Deployment
2. Configure Readiness and Liveness Probes
3. Use Pod Disruption Budgets
4. Node Affinity and Anti-Affinity
5. Configure Rolling Updates

By following these methods, you can make your applications more resilient to node updates and failures, ensuring continuous availability and minimal disruption to services.

1. Multi Replica Deployment for High Availability

Multi-replica deployment refers to the strategy of running multiple instances (replicas) of a pod within a Kubernetes cluster. Each pod is a running instance of a containerized application, and having multiple replicas means there are several copies of the same application running simultaneously (on same node or different nodes).

Benefits

1. High Availability:
   Redundancy: By running multiple replicas, the application remains available even if one or more pods fail. The failure of a single pod does not impact the overall availability of the service.
   Load Balancing: Kubernetes automatically distributes traffic across all available replicas, balancing the load and improving the responsiveness of the application.
2. Scalability:
   Horizontal Scaling: Adding more replicas allows the application to handle increased load. Kubernetes can dynamically adjust the number of replicas based on the current demand.
   Performance Improvement: With more replicas, the application can process more requests concurrently, enhancing overall performance.

Configuration

step-1: Define the Deployment
A Kubernetes deployment specifies the desired state of application instances. Here’s a sample YAML file for a multi-instance deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

In this example, the replicas: 3 field ensures that three instances of nginx are running.

step-2: Create a Service for Load Balancing
A Kubernetes service routes traffic to the multiple instances. Here’s a YAML file for a service:

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: LoadBalancer

2. Configure Readiness and Liveness Probes

Readiness and liveness probes are essential mechanisms in Kubernetes for monitoring and managing the health and status of application containers running in pods.

Readiness Probes

Readiness probes check whether an application is ready to accept incoming traffic. If a readiness probe fails, the pod is removed from the pool of endpoints that can receive traffic, preventing requests from being sent to an unready pod.

Use Cases:

• Startup Latency: Handle scenarios where an application needs time to load or initialize resources.
• Dependency Checks: Ensure that an application only starts receiving traffic once all necessary dependencies (e.g., databases, external services) are available.

Configuration

Readiness probes can also be configured using HTTP requests, TCP connections, or command execution
HTTP Readiness Probe:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: example-container
    image: example-image
    readinessProbe:
      httpGet:
        path: /ready
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 10

Let’s assume you have a web application running inside a container. This application has a specific endpoint (/ready) that returns a 200 OK status when the application is fully initialized and ready to handle requests. Here’s how the readiness probe would work in this case:

• k8s starts the container and waits for the initialDelaySeconds .
• send HTTP GET request to /ready endpoint.
• If the response is 200 OK, k8s marks the pod as ready.
• If the response is not 200 OK, k8s keeps checking at the specified intervals and does not route traffic to the pod until a successful response is received.

Similar to this we can use TCP Readiness Probe:

readinessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

and Exec Readiness Probe:

readinessProbe:
  exec:
    command:
    - cat
    - /tmp/ready
  initialDelaySeconds: 5
  periodSeconds: 10

Liveness Probes

Liveness probes check whether an application inside a pod is running as expected. If a liveness probe fails, Kubernetes assumes the container is in a bad state and attempts to recover it by restarting the container.

Use Cases:

• Application Deadlock: Detect if an application is stuck and unable to recover on its own.
• Critical Errors: Identify if an application has encountered an irrecoverable error state.

Configuration

Liveness probes can be configured using different methods such as HTTP requests, TCP connections, or command execution
HTTP Liveness Probe:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: example-container
    image: example-image
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5

TCP Liveness Probe:

livenessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

Exec Liveness Probe:

livenessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy
  initialDelaySeconds: 10
  periodSeconds: 5

Readiness and liveness probes are critical for maintaining the health and reliability of applications in Kubernetes. They enable automatic detection and recovery from failures, ensuring that only healthy and ready pods receive traffic. Properly configuring these probes helps maintain robust and resilient applications within a Kubernetes cluster.

3. Pod Disruption Budgets

Pod Disruption Budgets (PDBs) are a Kubernetes feature that ensures a certain number or percentage of pods in a deployment, replica set, or stateful set remain available during voluntary disruptions. Voluntary disruptions include events such as node drains for maintenance, cluster upgrades, or manual interventions. PDBs help maintain application availability and stability by controlling how many pods can be taken down simultaneously.

Benefits of Pod Disruption Budgets

• High Availability: PDBs help maintain application availability by preventing too many pods from being disrupted simultaneously.
• Controlled Maintenance: Administrators can perform maintenance tasks (like node upgrades) without impacting application availability beyond acceptable limits.
• Enhanced Stability: By ensuring a minimum number of pods are always available, PDBs contribute to the overall stability and reliability of the application.

Limitations

• Involuntary Disruptions: PDBs do not protect against unexpected failures such as node crashes or network issues. They are designed for managing voluntary disruptions.
• Complexity in Large Clusters: In very large clusters with many PDBs, managing and coordinating disruptions can become complex, requiring careful planning and monitoring.

Configuration

A PDB is defined by specifying the minimum number of pods that must be available during a disruption or the maximum number of pods that can be disrupted simultaneously. This is done using either minAvailable or maxUnavailable.

For example:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: example-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: example-app

In this example:

• minAvailable: 2 ensures that at least two pods must remain available at all times.
• selector: specifies the set of pods the PDB applies to, identified by the label app: example-app.

Alternatively, you can use maxUnavailable to specify the maximum number of pods that can be disrupted:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: example-pdb
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: example-app

In this example:

• maxUnavailable: 1 ensures that no more than one pod can be disrupted at any given time.

Pod Disruption Budgets are a powerful tool in Kubernetes for managing the availability of applications during planned disruptions. By defining the acceptable limits for pod availability, PDBs ensure that maintenance activities and updates can be performed without significantly impacting the application’s service level. Proper use of PDBs helps maintain high availability, enhance stability, and facilitate smoother operations in a Kubernetes cluster.

4. Node Affinity and Anti-Affinity

Node affinity and anti-affinity in Kubernetes are mechanisms used to control how pods are assigned to nodes based on the characteristics and labels of the nodes. These features help ensure that pods are scheduled in a way that meets the requirements of the application, such as optimizing resource usage, ensuring high availability, or adhering to specific operational policies.

Node Affinity

Node affinity allows you to constrain which nodes your pods can be scheduled on based on node labels. It is specified in the pod specification and provides more flexible and powerful placement rules compared to traditional node selectors.

Types of Node Affinity:

1. RequiredDuringSchedulingIgnoredDuringExecution: Pods must be scheduled on nodes that satisfy the affinity rules. This is a hard requirement.
2. PreferredDuringSchedulingIgnoredDuringExecution: Pods are preferred to be scheduled on nodes that satisfy the affinity rules, but it is not a hard requirement. If no preferred nodes are available, the pod can still be scheduled on other nodes.

Example Configuration:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: disktype
            operator: In
            values:
            - ssd
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          preference:
            matchExpressions:
            - key: region
              operator: In
              values:
              - us-west-1

In this example:

• Required Affinity: The pod can only be scheduled on nodes with the label disktype=ssd.
• Preferred Affinity: The pod prefers to be scheduled on nodes with the label region=eu-central-1, but it can be scheduled on other nodes if necessary.

Node Anti-Affinity

Node anti-affinity ensures that certain pods are not scheduled on the same nodes or specific groups of nodes. This is useful for spreading pods across different failure domains or avoiding resource contention.

Types of Node Anti-Affinity:

1. RequiredDuringSchedulingIgnoredDuringExecution: Pods must not be scheduled on nodes that satisfy the anti-affinity rules. This is a hard requirement.
2. PreferredDuringSchedulingIgnoredDuringExecution: Pods are preferred not to be scheduled on nodes that satisfy the anti-affinity rules, but it is not a hard requirement.

Example Configuration:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: disktype
            operator: NotIn
            values:
            - hdd
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - example-app
            topologyKey: "kubernetes.io/hostname"

In this example:

• Node Affinity: The pod cannot be scheduled on nodes with the label disktype=hdd.
• Pod Anti-Affinity: The pod prefers not to be scheduled on the same node as other pods with the label app=example-app. This helps distribute the pods across different nodes to enhance fault tolerance and resource utilization.

5. Configuring Rolling Updates

Rolling updates are a deployment strategy in Kubernetes that allow for updating applications without downtime. During a rolling update, Kubernetes incrementally replaces the old version of a deployment’s pods with the new version, ensuring continuous availability. This strategy is commonly used to deploy new versions of applications, apply configuration changes, or update dependencies.

Key Concepts

1. Deployment: A Kubernetes resource that manages a set of replica pods and ensures they are running and updated correctly.
2. ReplicaSet: A resource that maintains a stable set of replica pods running at any given time.
3. Rolling Update Strategy: The default update strategy for deployments, which replaces pods in a controlled, incremental manner.

Rolling Update Configuration

Rolling updates are configured in the deployment spec. The key parameters for controlling a rolling update are maxUnavailable and maxSurge.

• maxUnavailable: Specifies the maximum number of pods that can be unavailable during the update process.
• maxSurge: Specifies the maximum number of pods that can be created above the desired number of pods during the update process.

These parameters can be specified as either an absolute number or a percentage of the total number of pods.

Example Configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-deployment
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 2
  template:
    metadata:
      labels:
        app: example-app
    spec:
      containers:
      - name: example-container
        image: example-image:v2

In this example:

• replicas: 5: The deployment maintains five replicas of the pod.
• maxUnavailable: 1: During the update, at most one pod can be unavailable.
• maxSurge: 2: At most two additional pods can be created temporarily during the update.

Conclusion

Define a Highly Available Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-deployment
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 2
  selector:
    matchLabels:
      app: example-app
  template:
    metadata:
      labels:
        app: example-app
    spec:
      containers:
      - name: example-container
        image: example-image:v2
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              preference:
                matchExpressions:
                - key: disktype
                  operator: In
                  values:
                  - ssd
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - example-app
              topologyKey: "kubernetes.io/hostname"

• Multi-Replica: This ensures that 5 instances of the application are running, providing redundancy and load balancing.
• Rolling Update Configuration: The deployment uses a rolling update strategy with maxUnavailable set to 1 and maxSurge set to 2.
• Node Affinity: Pods are preferred to be scheduled on nodes with SSDs.
• Pod Anti-Affinity: Ensures that no two pods of this deployment are scheduled on the same node, promoting distribution across nodes.

Define LoadBalancer Service Configuration

apiVersion: v1
kind: Service
metadata:
  name: example-svc
spec:
  type: LoadBalancer
  selector:
    app: example-app
  ports:
    - protocol: TCP
      port: 80  # The port on which the service will be exposed
      targetPort: 8080  # The port on which the backend pods are listening

Define a Pod Disruption Budget (PDB)

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: example-pdb
spec:
  minAvailable: 3
  selector:
    matchLabels:
      app: example-app

Configure Node Labels

Ensure nodes have the necessary labels to work with the node affinity rules. For example, label nodes with the disktype key.

kubectl label nodes <node-name> disktype=ssd

By combining multi replica, rolling updates, pod disruption budgets, node affinity, and anti-affinity, you can create a highly available, resilient, and efficient deployment in Kubernetes. These configurations ensure continuous availability during updates, optimal resource utilization, and enhanced fault tolerance, contributing to the overall reliability and performance of your application.

Reference

• https://kubernetes.io/docs/tutorials/kubernetes-basics/scale/scale-intro/
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
• https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
• https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-intro/
• ChatGPT