Ultimate Kubernetes Infrastructure Monitoring — Metrics + Logs

Published in

Zenergy

3 min readMay 5, 2019

We have used helm for most of the deployments. Helms makes it easy to define, install and upgrade complex Kubernetes Application.

1. Installing Prometheus Operator and Grafana (Metrics Monitoring)

helm install \
 --name prom \
 --namespace monitoring \
 stable/prometheus-operator# pass -f values.yaml for custom configuration

You can change configurations in values.yml like storagesize and add additionalScrapeConfigs for monitoring other infrastructure if ServiceMonitor is not used.

This helm chart installs all default dashboards for your basic infrastructure and application monitoring.

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

github.com

2. Deploying EFK on Kubernetes (Centralised Logging)

# Add helm charts repository
helm repo add akomljen-charts https://raw.githubusercontent.com/komljen/helm-charts/master/charts/# Install Elasticsearch Operator
helm install --name es-operator \
 --namespace logging \
 akomljen-charts/elasticsearch-operator# Install EFK Stack for getting logs from containers and visualise in Kibana
helm install --name efk \
 --namespace logging \
 akomljen-charts/efk# pass -f values.yaml for custom configuration

Get Kubernetes Logs with EFK Stack in 5 Minutes

Last week I wrote about getting Kubernetes cluster metrics with Prometheus. Metrics are only one part of the story…

akomljen.com

3. Deploying Kong API Gateway for managing all dashboards using Proxy with Authentication

Using Kong removes the need for using static IP address with LoadBalancer for each service and also gives a centralised authentication system for all services.

helm install --name kg --namespace kong stable/kong# pass -f values.yaml for custom configuration

4. Install Konga for configuring Kong using Admin Dashboard

kubectl apply -f konga-cluster.yaml

5. Create services for accessing dashboard in Kong

Managing microservices and APIs with Kong and Konga

Introduction, setup and a basic use case example

medium.com

Other Commands

1. Updating services using helm

# updating prometheus-operator
helm upgrade \
 --namespace monitoring \
 prom \
 stable/prometheus-operator# updating efk
helm upgrade --namespace logging efk akomljen-charts/efk# updating kong
helm upgrade --namespace kong kg stable/kong

2. Check status of helm releases

kubectl --namespace monitoring get pods -l "release=<release_name>"

3. Dashboards port-forward

# port-forward grafana dashboard
kubectl port-forward svc/prom-grafana 3000:80 -n monitoring# port-forward prometheus dashboard
kubectl port-forward svc/prom-prometheus-operator-prometheus 9090:9090 -n monitoring# port-forward kibana
kubectl port-forward svc/efk-kibana 5601:80 -n logging# port-forward konga
kubectl port-forward svc/konga-svc 1337:1337 -n kong

4. Clean-up

# delete prometheus
helm delete --purge prom# delete efk
helm delete --purge efk
helm delete --purge es-operator# delete kong
helm delete --purge kg# remove konga
kubectl delete -f konga-cluster.yaml