Postgres on Kubernetes(K8s)

In today’s era of cloud-native computing, Kubernetes has become the go-to platform for deploying and managing containerized applications at scale. As organizations strive for flexibility, scalability, and resilience in their data management solutions, PostgreSQL, an open-source relational database management system, stands out as a popular choice for storing and manipulating structured data. When combined with Kubernetes, PostgreSQL offers a powerful solution for building resilient, highly available, and dynamically scalable database environments.

In this guide, we will explore the essential steps required to run PostgreSQL, a powerful open-source relational database management system, on Kubernetes.

postgres-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    group: db
spec:
  type: ClusterIP
  selector:
    app: postgres
  ports:
    - port: 5432
      targetPort: 5432

postgres-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  labels:
    group: db
data:
  POSTGRES_DB: hive_metastore
  POSTGRES_USER: hive
  POSTGRES_PASSWORD: hivepass123

postgres-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: spark-apps
  labels:
    app: postgres
    group: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
        type: db
    spec:
      volumes:                         # indicates which PVC are available for this Deployment
        - name: "postgres-storage"
          hostPath:
            path: "/Users/user/Documents/docker/osdsk8s/postgres-data"
            type: Directory
      containers:
        - name: postgres
          image: postgres:16
          ports:
            - containerPort: 5432
          envFrom:
            - configMapRef:
                name: postgres-config
          volumeMounts:                 # indicates which Volume (from spec.template.spec.volumes) should be used
            - name: postgres-storage                # name of the Volume
              mountPath: /var/lib/postgresql/data   # path inside the container
          resources:
            limits:
              memory: "256Mi"
              cpu: "500m"

This YAML configuration describes a Kubernetes Deployment resource for running a PostgreSQL database instance. Let’s break down the key components:

1. apiVersion: Specifies the API version of the Kubernetes resource. In this case, it’s apps/v1, indicating the version of the Deployment API being used.
2. kind: Specifies the type of Kubernetes resource. Here, it’s a Deployment, which manages a set of identical pods and ensures that a specified number of them are running.
3. metadata: Contains metadata about the Deployment, such as its name, namespace, and labels. Labels are key-value pairs that are used to organize and select resources. In this example, the Deployment is named “postgres”, belongs to the “spark-apps” namespace, and has labels indicating that it’s part of the “db” group and is associated with the “postgres” application.
4. spec: Defines the desired state for the Deployment, including the number of replicas (instances) to run, the selector to determine which pods the Deployment manages, and the pod template.

• replicas: Specifies the desired number of replicas (instances) of the PostgreSQL pod. In this example, it’s set to 1, indicating a single instance.
• selector: Specifies the labels used to select the pods that the Deployment manages. In this case, it selects pods with the label app: postgres.
• template: Defines the pod template used to create new pods controlled by the Deployment.
• metadata: Contains labels for the pod template, which are used for identifying and selecting pods.
• spec: Specifies the specification for the pods created by the Deployment.
• volumes: Defines the volumes available for the pod. In this example, it specifies a hostPath volume named “postgres-storage” with a path on the host machine where PostgreSQL data will be stored.
• containers: Describes the containers to run in the pod.
• name: Specifies the name of the container, which is set to “postgres”.
• image: Specifies the Docker image to use for the container, in this case, postgres:16.
• ports: Specifies the ports to expose on the container. Here, port 5432, the default PostgreSQL port, is exposed.
• envFrom: Specifies environmental variables for the container, sourced from a ConfigMap named “postgres-config”.
• volumeMounts: Defines the volumes to mount into the container. In this example, it specifies that the “postgres-storage” volume should be mounted at the path “/var/lib/postgresql/data” inside the container.
• resources: Specifies resource limits for the container, including memory and CPU limits. Here, the container is limited to 256MiB of memory and 500 milliCPU.

This below command will apply the configurations defined in postgres-svc.yaml, postgres-config.yaml, and postgres-deployment.yaml to your Kubernetes cluster

kubectl apply -f postgres-svc.yaml,postgres-config.yaml,postgres-deployment.yaml

To test, use the kubectl port-forward command:

kubectl port-forward deployment/postgres 5432:5432 -n spark-apps

Try connect from any SQL Client like DBeaver using data from postgres-config.yaml.

In the below section, instead of ConfigMap, use the Secret resource. Secrets are preferred over ConfigMaps for storing sensitive information such as passwords, API tokens, and other confidential data. Unlike ConfigMaps, which are primarily designed for storing configuration data, secrets are specifically encrypted and base64-encoded to ensure secure handling of sensitive information. By using secrets, you can better protect sensitive data from unauthorized access or exposure.

Here’s how you can create a secret with the equivalent data to your provided ConfigMap:

postgres-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
  labels:
    group: db
type: Opaque
data:
  POSTGRES_DB: aGl2ZV9tZXRhc3RvcmU=    # Base64 encoded value of "hive_metastore"
  POSTGRES_USER: aGl2ZQ==                # Base64 encoded value of "hive"
  POSTGRES_PASSWORD: aGl2ZXBhc3MxMjM=    # Base64 encoded value of "hivepass123"

kubectl apply -f postgres-secret.yaml

To use the secret in your Deployment, you need to reference it in the spec.template.spec.containers.env section of your Deployment YAML. Here's how you can modify your Deployment YAML to use the secret:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: spark-apps
  labels:
    app: postgres
    group: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
        type: db
    spec:
      volumes:
        - name: "postgres-storage"
          hostPath:
            path: "/Users/user/Documents/docker/osdsk8s/postgres-data"
            type: Directory
      containers:
        - name: postgres
          image: postgres:16
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_DB
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_DB
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_USER
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: POSTGRES_PASSWORD
          volumeMounts:
            - name: postgres-storage
              mountPath: /var/lib/postgresql/data
          resources:
            limits:
              memory: "256Mi"
              cpu: "500m"

By following the steps outlined in this guide, including creating services, configuration maps, and deployments, users can efficiently deploy and manage PostgreSQL instances on a Kubernetes cluster.