Guide to Log Analysis with Elastic Stack ( Part -1 )
What is Elastic Stack?
Elastic stack is an open-source software which is designed by Elastic. Elasticsearch, Logstash and Kibana are the three components of the Elastic Stack, which makes Elastic Stack a versatile tool that can be used for various use cases including log analysis. The three components of the ELK are designed to work interactively, which allows for the simplified collection, processing and analysis of data
Logstach
Logstach is a data processing tool primarily used to collect, process and transform data from various sources and prepare them to store in Elasticsearch
This is designed to handle various data types such as logs, metrics, events and other structured and unstructured data. Logstach provides a massive number of inputs, filters and output plugins that can be used to collect data from various sources transform it, and send it to desired destinations.
Elasticsearch
Elasticsearch is an open-source, distributed search and analytics engine software component that serves as the central component of the stack, that is used to store and index large volumes of data. this highly scalable system store and search various data types such as structured, semi-structured and unstructured data. Elasticsearch is built on top of Apache Lucene, which is an open-source search engine library.
Kibana
This is the user interface or the front end of the stack. Kibana provides a user-friendly interface, which allows users to create user-interactive dashboards, visualizations and reports based on the data stored in the Elasticsearch.
Kibana facilitates data store within the Elasticsearch letting users gain insights and find patterns. Combination of the three components Kibana is a crucial solution for log analysis and making sense in large volumes of data
Basics in Kibana
Discover
As you log in to Kibana, it will direct you to the home dashboard, which shows some options for managing Elastic Stack. Now let’s move on to the discover page, which you can find in the top left corner of the menu bar, analytics tab.
Discover is the tab that allows users to explore, search, and analyze data in real-time. This is a very powerful tool that allows users to find anomalies, patterns, and trends based on the log data. in order to do that users can search for specific terms, fields, and patterns in the data and also add filters to filter out the results based on the user criteria.
Time Filter
Endpoints and systems create massive amounts of logs, so it is impossible to monitor each log individually. Kibana’s filters make this task easy by filtering the results. The user-friendly time filter, allows users to filter out the results to a specific time range, making it easier to identify the events leading up to the incident.
Important — Kibana uses your local machine browser’s local time.
Users can edit the timezone settings here
Fields and Values of Kibana
agent.name: The field that indicates the software agent or workstation name responsible for generating the log event.
process.command_line: the field that captures the entire command-line string of the process. can use this field to search for a specific command that the attacker executed.
event.category: the field that indicates the nature of the event related to process execution.
process.name: the field that captures the name of the process.
process.parent.name: the field that captures the name of the parent process name.
process.executable: the field that captures the name or the executable file associated with the process.
source.ip: the field that captures the source ip address associated with a network event.
destination.ip: the field that captures the destination ip address associated with a network event.
related.user: the field that captures information about the user associated with the event.
All the above-mentioned selectable fields are available under the Available fields header on the left panel in the kibana. find the field the user wants to add to the field as a visual column. Go to the desired field hover over it and click the blue plus icon to add the field as the visual column. Users can also search the field directly by typing the field name on the search bar in the left corner panel in Kibana.
Sorting and Filtering in Kibana
Sorting in Kibana
Users can rearrange the order of the logs to show the users everything from the beginning of the selected time range, in ascending order. First, click on the sort icon next to the column tab and choose the Old-New.
Then the logs will be displayed from the earliest to the latest timestamp.
Field Filtering in Kibana
Let’s use the process.command_line field to filter out the results to look at the commands that were run on the compromise station. To accomplish this task click on the process.command_line field and select the “Filter from field present”. Then the filter will filter out the results and show only the fields containing the values in the selected field. Additionally, this allows users to see log events that contain specific information about the commands that were run.
In addition, we can isolate the workstation by going to the agent.name field and click on the blue plus icon in front of the desired workstation the user wants to isolate. Users can follow the same steps to isolate the related.user account.
