A Guide to Deploy Elasticsearch Cluster on Google Kubernetes Engine
This is a simple guide that helps you to deploy Elasticsearch cluster on Google Kubernetes Engine. This guide should be relevant on any Kubernetes cluster with a simple tweak on the persistent volume part.
The overall step-by-step is like the following:
- Enable the persistent volume via Storage Classes.
- Enable the Elasticsearch node discovery via Headless Service.
- Deploy the Elasticsearch Cluster via Stateful Sets.
1. Persistent Volume
The first step is to create a Storage Class on your cluster. Create new file called
storage.yaml with the following content:
Note that the Storage Class
provisioner that we use is GCE (
kubernetes.io/gce-pd ) with the following parameters:
type: pd-ssdto enable the SSD as persistent disk. You may update the parameters type with
type: pd-standardto enable the standard disk as persistent disk
zone: asia-southeast1-afor the compute zone. You may update this to your compute zone. You can also use
zonesparameter for multiple zones like the following:
zones: asia-southeast1-a, asia-southeast1-b.
Then enable the storage class using the following command:
kubectl apply -f storage.yaml
You should be able to see the Storage Class on the dashboard:
If you deploy the Elasticsearch Cluster on other than Google Kubernetes Engine, please update the provisioner and the parameters. You can see the available provisioners and parameters in here. The rest of this guide will applicable to any Kubernetes Cluster.
2. Elasticsearch Node Discovery
The second step is to enable elasticsearch node discovery via headless service. Create new file called
service.yaml with the following content:
- port: 9200
- port: 9300
Then enable the headless service using the following command:
kubectl apply -f service.yaml
You should be able to see the service on the dashboard:
Now, every pod that have label
service: elasticsearch should be accessible via
$PODNAME.es.default.cluster.local inside a kubernetes cluster. This will helps our elasticsearch nodes to discover each other and form a cluster.
3. Elasticsearch Cluster
The last step is to deploy the elasticsearch cluster using the StatefulSet. Create new file called
elasticsearch.yaml with the following content:
to deploy the Elasticsearch cluster, run the following command:
kubectl apply -f elasticsearch.yaml
It may takes time before the cluster is ready, it depends on the size of the cluster. You can see if the cluster is ready or not by accessing the workloads dashboard.
If your cluster is ready, you can check if cluster is created or not by accessing one of the elasticsearch node via port-forward:
kubectl port-forward elasticsearch-0 9200:9200
this will forward all request to http://localhost:9200 to the elasticsearch-0 node. Then:
It should show you that the cluster is formed by 5 elasticsearch nodes.
That’s it! now you can enjoy my favorite gif: