Install Elastic Stack on Ubuntu 16.04
Let’s talk about Elastic Stack
Elasticsearch is a search engine based on Lucene. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is developed in Java and is released as an open source software. Elasticsearch is the heart of « Elastic Stack ».
Logstash is an open source tool for managing system events and logs. It provides real-time pipelining to collect data. Logstash will collect the log or data, convert all data into JSON documents, and store them in Elasticsearch.
Kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.
Required
The tutorial can be run on other Lunix/Unix systems, most of the time you just need to change how packages are installed.
How to install Elastic Stack: step by step !
Installation of Java
Elasticsearch is working using Java, so we need to install the prerequisites, let’s start by installing it:
We need to add now the java repository:
Let’s install Java using the ppa webup8 repo:
Check the version.
Installation and configuration of Elasticsearch
Before installing Elasticsearch, we need to add the key of the repository :
Add the repertory:
Update and install.
Elasticsearch is now installed, we just need to configure it:
Activate the memory lock for Elasticsearch by removing the comment on line 43.
In the Network part, uncomment these lines:
Save and quit.
Now we need to edit the Elasticsearch service file:
Uncomment this line:
Save and quit.
Let’s edit the Elasticsearch configuration file in /etc/default:
uncomment this line:
Save and quit.
Now we start Elasticsearch:
Wait a little because Elasticsearch needs time to start, we can see now:
You should see this now:
Let’s check if Elasticsearch listens corectly on the port 9200.
The installation of Elasticsearch is now done, we can install Kibana now.
Installation and configuration of Kibana with Nginx
Let’s install Kibana with this:
We edit the configuration file:
Uncomment and edit:
Save and quit.
We can launch Kibana.
The Kibana port is 5601, we can check it.
We should see:
The installation of Kibana is now done, we can start the installation and configuration of Nginx.
Nginx is now installed, let’s create the Kibana configuration file for Nginx.
Copy the configuration below:
Save and quit. Write on your terminal:
Enter your password, it will be used for Kibana login page.
Let’s activate the VHost using a symbolic link.
We can test Nginx configuration and launch it.
Nginx is now ready for use, let’s install Logstash
Installation and configuration of Logstash
Install Logstash with the command:
Edit the hosts file.
Add the IP of your server.
Save and quit. We need to add now the SSL certificate.
If your server name is different from here, change the /CN value by your server name.
Let’s create now, the configuration files needed by Logstash.
Here is the entry configuration of Logstash:
Logstash will listen on port 5443. Save and quit.
We create now our syslog filter.
Put this inside your file:
Let’s finish by creating the output configuration.
Add this:
Save and quit.
Start Logstash.
Installation and configuration of Filebeat on our client Ubuntu
Connect to your server using SSH
Copy the logstash certificate on your server using scp.
Go inside the hosts file.
Add the IP of your master.
We need to add the Elasticsearch Key on our server.
We have to download the packages necessary to the Filebeat functioning.
Add the Elasticsearch repository.
Filebeat is now ready to be installed.
Let’s go to the Filebeat configuration.
Add the log you want to send.
Put the document type on syslog.
Disable the output of Elasticsearch by commenting these lines:
Enable Logtash output.
Save and quit.
Do not forget to add the certificate you took using scp.
Now we can launch Filebeat.
Check if Filebeat is active.
Is Filebeat working? To know if Filebeat is sending correctly your logs, you can check it there:
You should normally see that Filebeat is sending your logs.
Now you can check your Kibana configuration too, go to localhost:5601
Your default index is named filebeat:
Choose create, and click on the star on the top
Let’s go to the discover category, you should now see your logs.
Here is an example of a JSON output of your log.
This marks the end of our tutorial!
Do you want to know more about Wassa?
Wassa is an innovative digital agency expert in Indoor Location and Computer Vision. Whether you are looking to help your customers to find their way in a building, enhance the user experience of your products, collect data about your customers or analyze the human traffic and behavior in a location, our Innovation Lab brings scientific expertise to design the most adapted solution to your goals.
Find us on:
