Hacking with kibana

Data visualization with elasticsearch and kibana

λ.eranga
λ.eranga
Feb 23 · 6 min read

About kibana

Kibana is the data vitalization tool on ELK stack. We can take data from elasticsearch and build various types of analytical graphs, charts, data tables and dashboards on kibana. You can find more information about kibana from here. In this post I’m gonna show about building different types of graphs and dashboards with kibana. Following are the steps to follow :)

1. Setup index pattern

First we need to setup elasticsearch index on kibana. I have an index call packets which keep network devices packet information. We can setup the index from Management section in kibana.

The packets index contains following fields. Based on these fields we can do various searches and build the graphs

2. Discover data

After creating an index pattern we can browse all the data in the index from the Discover section.

3. Search data

On discover section we can filter data based on the fields. It supports various lucene search queries such as wildcard filters, term filters, range filters etc. Please read more about lucene queries from here. If you want we can save these queries as well. After that based on the saved search queries we can build visualization graphs and charts.

4. Visualization

As mentioned previously kibana supports to build various types of graphs, charts, data tables etc. We can navigate to Visualize section select a visualization type.

5. Line chart

I have chosen line chart and selected the packets index. Then navigated to chart configuration view. On line chart we need to define Y-axis and X-axis. In Y-axis by default it take count. For X-axis we need to define aggregation, timestamp field and interval. There are different types of aggregations available. I have chosen Date Histogram aggregation with following configurations. After defining configurations we can save the graph.

We can further split this line chart based on the index fields. I’m gonna split the chart with company_id field. To do that we need to go for add sub-bucket and split-series sub-aggregation as Terms. Now the graph series will be split based on the company_id field in the index.

We can write customer filter queries and split the line charts. In here I’m gonna split the chart with two filter queries

company_id: "rahasak" AND verified: true
company_id: "creative" AND verified: true

To define these filter I need to go for add sub-bucket, split-series and select the sub-aggregation as Filters.

6. Vertical bar chart

Select vertical bar chart and define the X-axis aggregation as Date Histogram. It will display a vertical bar chart with packet count on Y-axis like below.

As same as we have split the line charts with fields and terms we can split the bar charts as well. I have used company_id term to split the chart in here.

Previously I have used Date histogram as the aggregation. It built the chart with timestamp field. Now I’m going to use Terms as the aggregation.

If we want, we can further split this chart with sub-aggregations. I have split the chart with verified term in here.

7. Pie charts

I have selected pie chart from visualization and defined Terms aggregation with company_id field. It will build a pie chart with different companies on the index.

We can further split this pie char with sub-aggregations. Select add sub-bucket, split-slices and define the Terms as device_id.

8. Searchable data table

Select searchable data table from visualization and defined the terms to split-rows with device_id. It will build a data table with different companies on the index and their packet count.

Main thing to notice here is, this table is searchable. We can search this table by using lucene queries. I have searched this table with following lucene query.

device_id: 10.* AND verified: true

I can further filter this data table with sub-aggregations. I have added filter to this data table with dst_address field. If want we can label this filters with custom name.

9. Matrix

We can use matrix to define text based outputs. This is useful to display counts on dashboards. I have defined matrix with filters as below. In here I haven’t specify any filtering conditions, so it displaying all the packet count.

Then I have added two more filters(verified: true and verified: false) to this matrix and labeled them like below.

10. Dashboard

We have built various types of visualizations now. Next things is to build dashboards with them. Select Dashboard section add a new dashboard. Then we can select the graphs that we need to add to the dashboard. We can drag them and resize them on the dashboard.

Then we can save this dashboard and share it. Main thing is we can make these dashboard as realtime by defining refreshing interval.

Following are some more dashboards that I have built with using various charts and data tables. Once you get used with kibana its really easy and fun to build the stuffs :)

Rahasak

Have less, be more

λ.eranga

Written by

λ.eranga

Scala, Golang with Vim and Hockey: What else does a man need to be happy :)

Rahasak

Rahasak

Have less, be more

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade