Welcome to the ELK stack-part 1

The ELK stack is a collection of three open-source products




We will discuss the introduction of each one of these……

Elasticsearch — The Amazing Log Search Tool

Elasticsearch is a NoSQL database that is based on the Lucene search engine. That might be confusing because we usually think of search as something that we do, not something that needs to be served. However,the reality is that search can be quite complex, and search servers have been developed in response to that fact.


Real-time data and real-time analytics
The ELK stack gives you the power of real-time data insights,with the ability too perform super-fast data extractions from virtually all structured or unstructured data sources. Real-time extraction, and real-time analytics. Elasticsearch is the engine that gives you both the power and the speed.
Scalable, high-availability, multi-tenant
With Elasticsearch, you can start small and expand it along with your business growth-when you are ready. It is built to scale horizontally out of the box.As you need more capacity, simply add another node and let the cluster reorganize itself to accomodate and exploit the extra hardware. Elasticsearch clusters are resilient,since they automatically detect and remove node failures. You can set up multiple indices and query each of them independently or in combination.
Full text search
Elasticsearch uses Lucene to provide the most powerful full-text search capabilities available in any open-source product. The search feartures come with multi-language support, an extensive query language, geolocation support, and context-sensitive suggestions, and autocompletion.
Document orientation
You can store complex, real world entities in elasticsearch as structured JSON documents. All fields have a default index, and you can use all the indices in a single query to get precise results in the blink of an eye.

Logstash — Routing Your Log Data

Logstash is a log pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets. This virtually any type of log that you manage:system logs, webserver logs, error logs and app logs. As administrators, we know how much time can be spent normalizing data from disparate data source.

With Logstash, it’s super easy to take all logs and store them in a central location.The only prerequisite is a java runtime, and it takes just two commands to get logstash up and running.

Think of Logstash as a pipeline for event processing:it takes precious little time to choose the inputs, configure the filters, and extract the relevant, high-value data from your logs.

Kibana — Visualizing Your Log Data

Kibana is your log-data dashboard. Get a better grip on your data stores with point-and-click pie charts, bar graphs, trendlines, maps and scatter plots.You can Visualize trends and patterns for data that would otherwise be extremely tedious to read and interpret.Eventually, each business line can make practical use of your data collection as you help them customize their dashboards.