Elasticsearch 8.x Deployment

What changed?

Elasticsearch recently release version 8.0 and it came with some cool new features such as

1. k-nearest neighbor (kNN) search
2. Storage savings for keyword, match_only_text, and text fields
3. Faster indexing of geo_point, geo_shape, and range fields
4. PyTorch model support for natural language processing (NLP)

You can read more about these here.

In this article, I'll focus on how the deployment has changed and

The biggest takeaway is that security features are enabled and configured by default in version 8.x

We’ll deploy a single-node cluster in this article and I have used a free tier GCP VM instance to deploy my elastic search cluster, if you need to know how to do this, please follow these two videos on my youtube channel to setup your lab

• How to configure a new VPC network in the google cloud platform
• How to create VM instances in GCP using instance templates

With our lab VM setup, let’s deploy our elastic search v8.x demo system

1. Update System

sudo apt-get update

2. Install wget if it is not on the system

sudo apt-get install wget -y

3. Download and install elasticsearch public signing key

wget -qO — https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

4. Install apt-transport-https package

sudo apt-get install apt-transport-https -y

5. Save directory definitions — I’ve highlighted the change in URL for 8.x

echo “deb https://artifacts.elastic.co/packages/8.x/apt stable main” | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list

6. Update and Install Elasticsearch and Kibana

sudo apt-get update && sudo apt-get install elasticsearch && sudo apt-get install kibana

In the deployment logs, you will see the following log lines, firstly, you will see that elastic search built-in superuser elastic is created by default, and next you will see the superuser password which you need to store for future use.

Creating elasticsearch group… OK
Creating elasticsearch user… OK
Unpacking elasticsearch (8.0.0) …
Setting up elasticsearch (8.0.0) …
 — — — — —- Security autoconfiguration information — — — — — — — — — 
Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : 5aKNDmXUOVu9P*Ne_5rS
If this node should join an existing cluster, you can reconfigure this with
‘/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node — enrollment-token <token-here>’
after creating an enrollment token on your existing cluster.
You can complete the following actions at any time:
Reset the password of the elastic built-in superuser with 
‘/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic’.
Generate an enrollment token for Kibana instances with 
 ‘/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana’.
Generate an enrollment token for Elasticsearch nodes with 
‘/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node’.
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -

This log output also shows some commands that might be of interest such as

• How to reset elastic built-in user password
• How to generate enrollment tokens

7. Elasticsearch configuration directory contents

If you cd into the directory for elastic search configuration which is by default /etc/elasticsearch, you will see an entry for a certs directory and a Keystore, if you cd into certs directory you will notice that it has the following certificates

• http_ca.crt: This is the self-signed ca certificate for elastic search
• http.p12: This is the certificate that is used to encrypt client communication such as communication between Kibana and elastic search.
• transport.p12: This is the certificate that is used to encrypt cluster communications.

Next, elastic search Keystore is used to store secrets information for example your p12 passwords, you can look at the contents of the Keystore by using the following command

/usr/share/elasticsearch/bin/elasticsearch-keystore list
## You will get the following information 
autoconfiguration.password_hash
keystore.seed
xpack.security.http.ssl.keystore.secure_password
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password

To view any secret for example the password to open my http.p12 file, type the following command

root@elk-1:/etc/elasticsearch/certs# /usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

To open the http.p12 file and look at its contents you can type

openssl pkcs12 -nokeys -info -in /etc/elasticsearch/certs/http.p12 

When prompted for an import password, type in the password that you got from Keystore.

For production cluster deployments you might need to replace ca and HTTP and transport certificates depending on your organization’s security policy with those from your enterprise CA but if this is just a lab system default self-signed certifcates are fine.

8. Next, let's have a look at the configuration file for our cluster

sudo su
nano /etc/elasticsearch/elasticsearch.yml
 # change cluster name
cluster.name: demo-elk
 # give the cluster a descriptive name
node.name: elk-1
# change network binding
network.host: 0.0.0.0

The section above is similar to how a v7.x cluster is configured but in the same elasticsearch.yml file towards the bottom you will see the following section

# — — — — — — — BEGIN SECURITY AUTO CONFIGURATION — — — — — — — —
#
# The following settings, TLS certificates, and keys have been automatically 
# generated to configure Elasticsearch security features on 21–02–2022 13:58:48
#
# — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
 enabled: true
 keystore.path: certs/http.p12
# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
 enabled: true
 verification_mode: certificate
 keystore.path: certs/transport.p12
 truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: [“elk-1”]
# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]
# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]
# — — — — — — END SECURITY AUTO CONFIGURATION — — — — — — — — — — —

Now, the section of elasticsearch.yml above has some key configurations that I will explain here

First, you can see x-pack security is enabled by default using this line in the elasticsearch.yml file, you can also see that security enrollments are enabled as well

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

Secondly, you will see that is certificates to be used for encryption with API clients such as Kibana, Logstash, are stored in http.p12 file

xpack.security.http.ssl:
 enabled: true
 keystore.path: certs/http.p12

Thirdly, the following code instructs the system that cluster communication is encrypted and the relevant certificates can be found in certs/transport.p12

xpack.security.transport.ssl:
 enabled: true
 verification_mode: certificate
 keystore.path: certs/transport.p12
 truststore.path: certs/transport.p12

Next, we provide a static set of initial master nodes which is by default only this node but if you are setting up a cluster you will edit this entry to include other initial master nodes.

cluster.initial_master_nodes: [“elk-1”]

Lastly, we specify which networks are HTTP and cluster communication allowed on

# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]
# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]

We will not make any changes to this section for our demo in this article.

8. Start Elasticsearch service

sudo systemctl start elasticsearch

9. Validate Elasticsearch cluster health

curl -k -u elastic:<password> https://localhost:9200/_cluster/health?pretty 
## -k to ignore ssl verification, if you are providing enterprise certificates you can remove it.

10. Next we need to set a password for the kibana_system user as we will need this information for communication between elastic search and Kibana

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana

11. Configure Kibana

nano /etc/kibana/kibana.yml
## uncomment server.port
server.port: 5601
## server base url however this needs to be corrected everytime you start and stop the server
server.publicBaseUrl: “http://<hostname_or_ip_address>:5601/"
## change server.host
server.host: “0.0.0.0”
## change server.name
server.name: “demo-kibana”
## uncomment elasticsearch.host
elasticsearch.hosts: [“https://localhost:9200"]
## add kibana_system user credentials
elasticsearch.username: "kibana_system"
elasticsearch.password: "<your_kibana_system_user_password>"

12. Start Kibana service

systemctl start kibana

13. Enable elastic search and Kibana

systemctl enable elasticsearch
systemctl enable kibana

Open Kibana by typing in the IP address of your elk and port 5601

https:\\<<ip_address>>:5601

and it should display the home page for Kibana.

If you enjoyed reading this article, please clap and follow me on medium.