Deploying Wazuh in Local Kubernetes Environment: Simplifying Security Monitoring

5 min readJul 17, 2024

As organizations increasingly adopt microservices architectures and containerization, the need for proactive security measures becomes more critical. Wazuh stands out as a versatile tool for real-time threat detection, integrity monitoring, and security analytics. By deploying Wazuh on Kubernetes, you can leverage Kubernetes’ inherent benefits of scalability and resilience while maintaining a vigilant stance against potential vulnerabilities and attacks. This tutorial will walk you through each step, from setting up Kubernetes environments to configuring Wazuh agents and managers, ensuring you can deploy and utilize Wazuh effectively within your Kubernetes cluster.

Here is the topology of Kubernetes Cluster that will be used in this tutorial:

Deploying Wazuh in Local Kubernetes Environment: Simplifying Security Monitoring — Topology

There are some pre-requisites needed before deploying wazuh:

The kubernetes cluster should have a non-default Storage Provisioner configured for Wazuh storages. In this tutorial, We’ll use NFS storage provisioner connected with NFS server
The kubernetes cluster should have load balancer service installed for Wazuh services. We use MetalLB in this tutorial

Do the deployment steps in a machine with kubectl configured to interact with the cluster. In this tutorial, It’ll be done in k8s-master with root user

A. Step by Step Deployment Guide

Install git and openssl

apt install git openssl -y

Create a new directory for operations. We’ll create a directory in /root/ called wazuh-deploy. Go into that directory and clone the stable branch of wazuh-kubernetes repository

mkdir /root/wazuh-deploy
cd /root/wazuh-deploy
git clone -b stable https://github.com/wazuh/wazuh-kubernetes

After that, generate certificate for Wazuh indexer cluster. The repository comes with self-signed certificates generation script. You can change and make needed adjustments to the script in wazuh-kubernetes/wazuh/certs/indexer_cluster/generate_certs.sh then execute it

bash wazuh-kubernetes/wazuh/certs/indexer_cluster/generate_certs.sh

Then, generate certificate for Wazuh dashboard HTTP service. The self-signed certificates generation script is also pre-generated. Feel free to change and adjusts based on needs. The script is located in wazuh-kubernetes/wazuh/certs/dashboard_http/generate_certs.sh. Execute it to generate the certificates

bash wazuh-kubernetes/wazuh/certs/dashboard_http/generate_certs.sh

If you are using your own generated certificates, store those certificates files in specific directories. Here are the needed certificate files with the directory structures:

wazuh-kubernetes
└── certs
    ├── indexer_cluster
    │   ├── root-ca.pem
    │   ├── root-ca-key.pem
    │   ├── node-key.pem
    │   ├── dashboard.pem
    │   ├── dashboard-key.pem
    │   ├── admin.pem
    │   ├── admin-key.pem
    │   ├── filebeat.pem
    │   └── filebeat-key.pem
    └── dashboard_http
        ├── cert.pem
        └── key.pem

Next, change the storage provisioner for deployer with configured provisioner in the cluster. In this tutorial, We use NFS storage provisioner. You can show available storage provisioners with this command:

kubectl get storageclass | awk '{ print $2 }'

After that, edit file wazuh-kubernetes/envs/local-env/storage-class.yaml

nano wazuh-kubernetes/envs/local-env/storage-class.yaml

Change the provisioner and save the file. For this tutorial, the provisioner is cluster.local/nfs-subdir-external-provisioner

provisioner: cluster.local/nfs-subdir-external-provisioner

After that, change all secrets that will be used. It’s a best practice not to use the default secrets. Note that all secrets in kubernetes YAML deployment is in base64 encoded cipher. So for your custom secret, Encode it first and get the encoded value.

echo 'YOUR_SECRET_HERE' | base64

Then insert the value to secret YAML file of Wazuh deployment. There are 5 secrets that’ll be used:
- Wazuh dashboard credential secret. This secret will be used for secure communications between the Wazuh dashboard and the Wazuh indexer
- Wazuh indexer credential secret. This secret will be used by the Wazuh manager to securely connect and send data to Opensearch. We’ll also use this credential as Wazuh admin login credential
- Wazuh API credential secret. This secret will be used to authenticate and authorize access to the Wazuh API
- Wazuh authd password secret. This secret will be used for authenticating communication between the Wazuh agents and the Wazuh manager
- Wazuh cluster key secret. This secret will be for secure communication and coordination between multiple Wazuh managers in a clustered environment

To edit the credential files, these are the command (Sequentially from Wazuh dashboard credential secret to Wazuh cluster key secret)

nano wazuh-kubernetes/wazuh/secrets/dashboard-cred-secret.yaml
nano wazuh-kubernetes/wazuh/secrets/indexer-cred-secret.yaml
nano wazuh-kubernetes/wazuh/secrets/wazuh-api-cred-secret.yaml 
nano wazuh-kubernetes/wazuh/secrets/wazuh-authd-pass-secret.yaml
nano wazuh-kubernetes/wazuh/secrets/wazuh-cluster-key-secret.yaml

After that, apply the Wazuh kubernetes deployment kustomization for local kubernetes environment

kubectl apply -k wazuh-kubernetes/envs/local-env

Wait for all pods in wazuh namespace to be in Running state

watch kubectl get pod -n wazuh

After everything is OK, Get the Wazuh indexer credential string. We’ll use this credential string as Wazuh admin login credential

for secret_type in username password; do secret=$(kubectl get secret -n wazuh indexer-cred -o yaml | egrep "$secret_type:" | awk '{print $2}'); echo "$secret" | base64 -d; echo ""; done

The first string is username and the second is password

Get the external IP address of Wazuh dashboard service

kubectl get service -n wazuh dashboard

Open the external IP address in browser to access the Wazuh Dashboard. The URL is using HTTPS.

https://<wazuh_dashboard_service_external_IP>

Once opened, Login to the Wazuh dashboard using Wazuh admin login credential that has been retreived before

After that, make sure the health checks are passed. Wazuh inside the Kubernetes Cluster is ready to be used

B. Some Troubleshootings for Errors

While pods deployment in process, You might encounter ImagePullBackOff error. If you describe the pod, You’ll see an event like this:

Failed to pull image "wazuh/wazuh-indexer:4.4.4": rpc error: code = Canceled desc = failed to pull and unpack image "docker.io/wazuh/wazuh-indexer:4.4.4": context canceled

This error happens because slow connectivity/process while downloading and unpacking the image and kubectl timeout values is exceeded.

To solve this problem, first get the node of that pod

 kubectl get pod -n wazuh <POD_NAME> -o wide

Then go to that node, and pull the image manually. Note the image name in the event message. Wazuh images are currently hosted in docker registry. Combine the registry name (docker.io) and image name

docker.io/<image_name>

Execute the pull operation and wait until finished

sudo -i
ctr -n k8s.io image pull docker.io/<image_name>

For example, to fix Wazuh indexer ImagePullBackOff, pull the image docker.io/wazuh/wazuh-indexer:4.4.4

ctr -n k8s.io image pull docker.io/wazuh/wazuh-indexer:4.4.4

After deployment, The Wazuh dashboard might in not ready state

This happens because the Wazuh dashboard service is not fully started yet. You can wait for the process to be fully started and watch the log

kubectl logs -n wazuh <wazuh_dashboard_pod_name> -f

It’s fully started when You saw logs similar to this:

Wazuh dashboard logs that indicating that It’s fully started

Then try to re-open the Wazuh dashboard.

References:

Deploying Wazuh in Local Kubernetes Environment: Simplifying Security Monitoring

Written by Kevin Tim