Prometheus with “kube-prometheus-stack”: Demystifying Kubernetes Monitoring — A Comprehensive Guide

Prometheus is a scalable and flexible monitoring solution, but navigating its configuration and usage can pose initial challenges. To overcome these hurdles and swiftly start monitoring your Kubernetes environment, there’s a resource called “kube-prometheus-stack.”

What is kube-prometheus-stack?

“kube-prometheus-stack” is a package that brings together Prometheus and other monitoring tools, optimized for use in Kubernetes environments. This repository offers a pre-configured solution, making it easier to use Prometheus within Kubernetes clusters.

Ready-to-Use Metrics:

This repo includes a pre-configured Prometheus installation that automatically collects metrics from Kubernetes resources (pods, services, storage, etc.). Additionally, the kube-prometheus-stack provides Grafana integration with ready-made dashboards and visualizations. This enables users to quickly assess the health of their applications and infrastructure.

Advantages of Usage:

• Swift Installation: kube-prometheus-stack facilitates a quick installation of Prometheus and other components.
• Easy Management: Configuration and management processes are automated with the Prometheus Operator.
• Grafana Integration: Ready-made dashboards and visualizations make understanding metric data more accessible.

Getting Started:

First, let’s create a Kubernetes cluster where we will install prometheus-community/kube-prometheus-stack helm chart. After setting up the Kubernetes cluster using kubeadm, AWS EKS, Azure AKS, or GKE, we can install prometheus-community/kube-prometheus-stack helm chart on the cluster following the steps I specify, once we deploy the sample application provided below.

1. Step-1: I will quickly set up Azure AKS using Azure CLI. You can also set it up in your preferred environment And, as an example, I’ve deployed two web applications along with MySQL to illustrate.

# creating resource group
az group create --name rg-mkanus --location eastus

# creating kubernetes cluster
az aks create --resource-group rg-mkanus --name mkanusm --location eastus --kubernetes-version 1.28.3 --tier standard --enable-cluster-autoscaler --node-count 2 --min-count 2 --max-count 3 --max-pods 110 --node-vm-size Standard_D2s_v3 --network-plugin azure --network-policy azure --load-balancer-sku standard --enable-addons monitoring --generate-ssh-keys

Step-2: Now that the Kubernetes cluster is ready with the applications deployed, we can proceed with the installation of kube-prometheus-stack

# Create a new namespace named 'monitoring' for kube-prometheus-stack installation
kubectl create namespace monitoring

# Get Helm Repository Info
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install Helm Chart setting "grafana.adminPassword=?"
helm upgrade --install prom-stack prometheus-community/kube-prometheus-stack --namespace monitoring --set grafana.adminPassword="mkanusm_123*"

• We installed the kube-prometheus-stack Helm chart using the commands provided above.

• After installing kube-prometheus-stack Helm chart, we can use the kubectl get all -n monitoring commmand to view all its components.

Step-3: You can find the port-forward commands for all the following “kube-prometheus-stack” services below.

# Port-forward commands for kube-prometheus-stack services

# Alertmanager
kubectl port-forward service/alertmanager-operated 9093:9093 -n monitoring &

# Grafana
kubectl port-forward service/prom-stack-grafana 3000:80 -n monitoring &

# Prometheus Alertmanager
kubectl port-forward service/prom-stack-kube-prometheus-alertmanager 9093:9093 -n monitoring &

# Prometheus Operator
kubectl port-forward service/prom-stack-kube-prometheus-operator 443:443 -n monitoring &

# Prometheus
kubectl port-forward service/prom-stack-kube-prometheus-prometheus 9090:9090 -n monitoring &

# kube-state-metrics
kubectl port-forward service/prom-stack-kube-state-metrics 8080:8080 -n monitoring &

# Prometheus Node Exporter
kubectl port-forward service/prom-stack-prometheus-node-exporter 9100:9100 -n monitoring &

# Prometheus Operated
kubectl port-forward service/prometheus-operated 9090:9090 -n monitoring &

• kubectl port-forward service/prom-stack-kube-prometheus-alertmanager 9093:9093 -n monitoring &

• kubectl port-forward service/prom-stack-kube-prometheus-prometheus 9090:9090 -n monitoring &

• kubectl port-forward service/prom-stack-kube-state-metrics 8080:8080 -n monitoring &

• kubectl port-forward service/prom-stack-grafana 3000-80 -n monitoring &

• By examining the metric values in the pre-packaged dashboard, we can visually inspect the metric values of all resources in the Kubernetes cluster.

Conclusion:

“kube-prometheus-stack” streamlines Prometheus usage and kickstarts monitoring with ready-to-use metrics. This repository serves as an ideal starting point for those considering performance monitoring in Kubernetes environments. Swift installation, easy management, and Grafana integration empower users to better understand and manage their applications and infrastructure.