A walk-through open Distro

8 min readApr 12, 2020

Welcome to our third article of this series. I encourage you all to check the first article to have a better understanding of what we are going to discuss here.

open Distro available plugins are :

Security
Alerting
SQL
Index State Management
Performance Analyzer

In our project we installed just the security and alerting plugins.

1- Alerting Feature :

Open Distro for Elasticsearch allows you to monitor your data and send alerts automatically to your stakeholders. It is easy to setup and manage and it uses Kibana interface with a powerful API.

Alerting feature allows you to setup rules so that you can be notified when something of interest changes in your data. Anything you can query on, you can build an alert on that. The Alerting feature notifies you when data from one or more Elasticsearch indices meets certain conditions.

This URL provides the versions history of opendistro ( in our case we will use 1.6.0 ) :

https://opendistro.github.io/for-elasticsearch-docs/version-history/

To add the alerting feature of opendistro you have to manage plugins for elasticsearch and kibana : managing plugins is under

/usr/share/elasticsearch : for Elasticsearch
/usr/share/kibana : for Kibana

1.1- Install Alerting plugin for elasticsearch :

cd /usr/share/elasticsearch
sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-sql/opendistro_sql-1.6.0.0.zip

1.2- Install the corresponding alerting plugin for kibana :

cd /usr/share/kibana
sudo bin/kibana-plugin install — allow-root https://d3g5vo6xdbdb9a.cloudfront.net/downloads/kibana-plugins/opendistro-alerting/opendistro-alerting-1.6.0.0.zip

1.3- To list your plugins or remove them , you can use these commands :

- For Kibana :

sudo bin/kibana-plugin list
sudo bin/kibana-plugin remove <plugin-name>

- For elasticsearch :

sudo bin/elasticsearch-plugin list
sudo bin/elasticsearch-plugin remove <plugin-name>

1.4- Now you should restart kibana and elasticsearch :

systemctl restart kibana elasticsearch

NOTE : after installing , removing or updating plugins for elasticsearch or kibana , it takes about few minutes to restart properly ,meanwhile your kibana server will be responding with ( kibana is not ready yet ). You could check kibana and elasticsearch processes in real time with command top.

1.5- Now you can check your plugins in kibana interface :

1.6- Working with alerting plugin :

a ) Creating Slack webhook URL :

Slack is a workplace communication tool, “a single place for messaging, tools and files.” This means Slack is an instant messaging system with lots of add-ins for other workplace tools.

Incoming Webhooks are a simple way to post messages from apps into Slack. Creating an Incoming Webhook gives you a unique URL to which you send a JSON payload with the message text and some options. You can use all the usual formatting and layout blocks with Incoming Webhooks to make the messages stand out.

First create an account ( slack.com)

Choose the second if you are new in Slack ( you don’t belong to any workspace )

Get the code that you just received in your mailbox and create your new workspace

Try to follow the steps until your reach your home page , Go to application and search for Incoming Webhook, click add :

Click add to Slack

Choose a channel to receive messages ( for example test ) and click add integration

Now scroll down until you find your Webhook URL ( save it because we will use it later )

Go to Kibana → Alerting → Destination and click add destination :

Choose Destination name , Pick Slack destination , Copy the your Webhook URl and click create

1.6.2- Creating Alerts and sending them to Slack :

Now go to Monitoring and click Create Monitor :

Setup your settings : you can use Graphical setup or extraction query setup

Here an example of Graphical Setup ( Event ID: 4624 means that an account has successfully logged on )

Check you Monitor Schedule and click create

Now you should create a trigger , For example :

Now go to notification and select the destination that you have created then click create :

Now you can follow your alerts in Kibana Alerting interface and every alert will be sent to your Slack Channel :

Go to your slack channel ( #test channel in this tutorial ) and wait for your alerts:

2- Security Feature :

This plugin provides a user interface for managing users, roles, mappings, action groups, and tenants.

2.1- Installing security plugin :

The choice of this pluging was made because Kibana does not come with an authentication panel in the basic version. Thus to ensure the security of our interfaces we will be using the free authentication panel offered by open distro.

You can follow the same steps ( from 1 to 4 ) as installing alerting plugin to install security plugins. You just have to change the URL install plugin :

For Kibana :

sudo bin/kibana-plugin install — allow-root https://d3g5vo6xdbdb9a.cloudfront.net/downloads/kibana-plugins/opendistro-security/opendistro_security_kibana_plugin-1.6.0.0.zip

For Elasticsearch :

sudo bin/elasticsearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/elasticsearch-plugins/opendistro-security/opendistro_security-1.6.0.0.zip

A warning message will be displayed : type y

After installing the Security plugin, you can run :

sudo sh /usr/share/elasticsearch/plugins/opendistro_security/tools/install_demo_configuration.sh

to quickly get started with demo certificates.

Otherwise, you must configure it manually and run securityadmin.sh.

We will use the first option :

cd /usr/share/elasticsearch/plugins/opendistro_security/tools/
change permission if required : chmod +x install_demo_configuration.sh
execute : ./install_demo_configuration.sh

For all question type y and the default credentials are ( Username : admin / Password: admin )

Another configuration will be added by open distro security in your /etc/elasticsearch/elasticsearch.yml

2.2- Changing configuration for elasticsearch logstash and kibana :

In our case we will make a username, password and SSL certificate for elasticsearch , we want to mention that the verification of the certificate in this section and in all other upcoming sections is beyond the scoop of these articles.

2.2.1- For Elasticsearch :

Disabling x-pack security feature for elasticsearch : When restarting elastic you will probably get an error because of xpack security feature enabled by default for the ELK Stack Basic Version , so you will have to disable it in /etc/elasticsearch/elasticsearch.yml before restarting your service.

2.2.2- For Kibana:

Disabling x-pack security feature for kibana : Also for kibana we have to disable the xpack.security feature and ignore ssl verification in /etc/kibana/kibana.yml

NOTE : Make sure that your protocol is https instead of http

2.2.3- For Logstash:

As our beats are not connected directly to elasticsearch , they are instead connected to logstash , so we don’t have to manage our beats or reconfigure them , we have only to configure logstash configuration file.

Make sure that your protocol is https instead of http

sudo nano /etc/logstash/conf.d/logstash.conf

NOTE : if you are reconfiguring beats or setting up another beat while your elasticsearch has the security plugin installed with username , password and SSL certificate you can add this configuration to your beat to make it reachable. Make sure that your protocol is https instead of http.

2.3- Resart all of your services :

systemctl restart elasticsearch
systemctl restart logtash
systemctl restart kibana

As mentioned before it may takes few minutes to restart properly and you check that with top command in real time. meanwhile your kibana server will be responding with ( kibana is not ready yet ).

Now your ELK stack is connected properly with new security credentials.

You can check that by using the Elasticsearch URL ( http will not work, you have to use https instead )

https://your_elasticsearch_ip:9200