What is ELK and Installing ELK stack (elasticsearch, logstash, kibana) in Ubuntu

What is ELK Stack

The ELK stack is a collection of three open-source tools — Elasticsearch, Logstash, and Kibana — that work together to collect, store, and analyze data. Here is how each tool works:

1. Elasticsearch: Elasticsearch is a search and analytics engine that provides a distributed, real-time search and analytics platform. It stores data in a distributed index, which allows for fast, real-time search and analysis of data. Elasticsearch is highly scalable, and it can handle a large volume of data with ease.
2. Logstash: Logstash is a data pipeline that collects, filters, and transforms data from different sources before sending it to Elasticsearch. It can collect data from different sources such as log files, databases, and message queues. Logstash can also perform data transformation and filtering to ensure that only the relevant data is sent to Elasticsearch.
3. Kibana: Kibana is a data visualization tool that allows users to create dashboards, visualizations, and reports based on data stored in Elasticsearch. Kibana provides a web interface that allows users to interact with the data and visualize it in various formats such as tables, graphs, and maps.

When used together, Elasticsearch, Logstash, and Kibana form a complete data analysis solution. Logstash collects data from different sources and filters and transforms it before sending it to Elasticsearch for storage.

Elasticsearch stores the data and provides fast, real-time search and analysis capabilities. Kibana provides a user-friendly interface to visualize and analyze the data stored in Elasticsearch.

Overall, the ELK stack is a powerful tool for collecting, storing, and analyzing data in real-time. It can be used for various use cases such as log analysis, security analysis, and business intelligence.

Installation and Configuration of ELK Stack on Ubuntu

Note: In the process of installation you can supply ubuntu machine IP in place of localhost or else you can also use localhost.

Install Java environment packages by using the below command

sudo su
apt install default-jdk default-jre -y

Add the elasticsearch APT repository key by using the below command

curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Add the elastic to the APT source list by using the below command

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list

Update the APT source list by using the below command

apt update

Install the Elastic Search by using the below command

apt install elasticsearch -y

Configure the elasticsearch by using the below command

nano /etc/elasticsearch/elasticsearch.yml

Change the network.host and http.port as per the screenshot(network.host is the IP of the Ubuntu machine)

Add “discovery.type: single-node”

Configure the JVM heap memory by using the below command

vim /etc/elasticsearch/jvm.options

Restart and enable elasticsearch.

systemctl restart elasticsearch
systemctl enable elasticsearch

To verify the working of elasticsearch use curl command as given below

curl -X GET "<IP>:9200"

Install the Logstash using apt

apt install logstash

Start and enable Logstash services

systemctl start logstash
systemctl enable logstash

Check the status of the Logstash Service

systemctl status logstash

Now install Kibana

apt install kibana

Configure kibana.yml file in /etc/kibana

nano /etc/kibana/kibana.yml

Start and enable kibana service

systemctl start kibana
systemctl enable kibana

Check the status of the kibana service

systemctl status kibana

Ping the http://<IP>:5601 or http://localhost:5601 in browser to view the Dashboard of the kibana as show in the below image.

You have successfully installed ELK stack in Ubuntu. In the upcoming blog I will share how to configure xpack security to enable minimal security.

You can find the installation video below:

Also check out my Masters in USA journey from the profile,

Rama Ramana Sharma — Medium

Subscribe to our YouTube channel. https://www.youtube.com/@CyberToolGuardian/featured

Follow us on Instagram.
https://instagram.com/cybertoolguardian