Deconstructing Elasticsearch 8

The “other” observability tool for Rancher and Kubernetes is Elasticsearch.

Here is a manual installation of the complete stack; as in the previous article on Prometheus, this installation neither provides high availability nor persistence.

All configuration files below have been copied and adapted from the respective Github repositories and follow their published licenses.

Kube State Metrics

To support metric collection, we will install kube-state-metrics first:

• 20-roles.yaml has the service account and role bindings
• 22-deployment.yaml has the actual deployment

kube-state-metrics depends on the Kubernetes version; check here to deploy the correct version.

Elasticsearch

The first piece of software we’ll install is Elasticsearch itself:

• 01-config.yaml has the configuration
• 02-deployment.yaml has the actual deployment

Security is explicitly disabled for this deployment; we do not expose the service outside the Kubernetes cluster. Also, check the version of the Elasticsearch images here before deploying.

Kibana

Elasticsearch uses Kibana for visualization:

• 12-deployment.yaml has the actual deployment

The deployment does not include dashboards or canvas; a good source is the Kibana demo instance or the Elastic content share.

Filebeat

To collect container and system logs, we’ll either install Filebeat, enable Rancher logging, or use both:

• 30-roles.yaml has the service account and role bindings
• 31-config.yaml has the configuration
• 32-deployment.yaml has the actual deployment

Metricbeat

To collect system and Kubernetes metrics, we’ll install Metricbeat:

• 40-roles.yaml has the service account and role bindings
• 41-config.yaml has the configuration
• 42-deployment.yaml has the actual deployment

Syslog forwarding

As an option, we can configure Rancher’s logging operator:

• 51-config.yaml has the cluster flow and output

Navigation Link

As a finishing touch, we will put a navigation link to Kibana into the Rancher UI:

• 99-navlink.yaml has the navigation links

A big shoutout to William Rizzo of SUSE and Emmet Bradley of BSH Container Services for their idea and inspiration!

All files are available on my Github, in Rancher/aks-cluster/elastic.

Happy Ranching!