Analytics Vidhya
Published in

Analytics Vidhya

Basics of Data Search in Elasticsearch

by Anurag Srivastava, Aug 4, 2018, 7:02:21 AM | 5 minutes

Day by day the volume of data is increasing as we are moving towards the digital age and transforming things into the Internet of things. Take a simple example of smartwatch what it does, it measures the steps, BP, heart rate, etc and pushing to a server from where we can access our health-related metrics. In the same way, there are different smart devices which keep on sending the regular data which is stored on a server. We are dumping lots and lots of data on servers which is there to help us find a trend, analyze them through data science to solve some serious problems or apply machine learning algorithms to forecast the future trend.

So in a brief way, I have explained how data is important and we should capture them regularly to get meaningful information out of it. Now the question arrives how. What is best way to search the data as traditionally we were storing data in a RDBMS system and use to fetch them by directly applying the SQL queries but now things have changed because we want quick search response, We have no time to wait for a search result by seeing the loading icon moving in a direction. Another issue which we were facing was the uncertainty of data format and for these types of data, we had a bottleneck in the RDBMS system.

Now move to the search part as this blog is here introduce you about the basic Elasticsearch query construction so that a novice person can install, index and search data from an Elasticsearch cluster. These days Elasticsearch is primarily used for its search capabilities and the ELK stack which can be applied to any set of application to boost the performance and monitoring capabilities. So let’s start the process and learn the basic search query construction in Elasticsearch.

We have basically two types of search APIs in Elasticsearch: ‘request URI based’ and ‘request body based’. In REST request URI we use to pass the search criteria in the URL itself like:

GET /blogs/technical/_search?q=topic:kibana

In REST request body we use to construct the search block and write the search query inside the query block of Elasticsearch like:

So the URI based search is quite a basic search where we just want to search a keyword whereas in request body we can construct the complex queries. So we have the query language to work with request body based searches. In this blog, I am not going into details to keep it simple so that everyone can understand what is going on.

Fielddata is disabled on text fields by default in Elasticsearch so we need to enable it for constructing the queries.

PUT blogs/_mapping/technical?update_all_types
{
"properties":
{
"topic":
{
"type": "text",
"fielddata": true
}
}
}

Now let’s understand the basics of the query language, where first comes the match_all query:

In match_all query, Elasticsearch returns all the documents. So this Elasticsearch query is basically like SQL “select * from technical” query.

Limit:

Now we are going to set the offset and limit in a query to restrict the records, like:

In the above query, I am fetching 5 documents starting from the second one. In the same way, we can set the offset and limit in any Elasticsearch query.

In Elasticsearch we can sort the documents as per our requirement like:

In the above expression, we are applying the ordering on the field topic.

We limit the number of columns in SQL select queries, in the same way, we can do it in Elasticsearch queries, like:

GET /blogs/technical/_search
{
"query": { "match_all": {} },
"_source": ["category"]
}

In the above query, we will only get the category field in search result and topic filed would not be displayed.

We can run the match queries against the field name, like:

In the above query, we can pass the text to search against the topic field.

In this blog, I have explained the basics of Elasticsearch query construction. I next blog I will cover filters, boolean queries, wild-card queries, etc and then will explain aggregation and its usage.

You can follow me on Twitter: https://twitter.com/anubioinfo

Other Blogs on Elastic Stack:
Introduction to Elasticsearch

Elasticsearch Installation and Configuration on Ubuntu 14.04
Log analysis with Elastic stack
Elasticsearch Rest API
Basics of Data Search in Elasticsearch
Elasticsearch Rest API
Wildcard and Boolean Search in Elasticsearch
Configure Logstash to push MySQL data into Elasticsearch
Metrics Aggregation in Elasticsearch
Bucket Aggregation in Elasticsearch
How to create Elasticsearch Cluster

If you found this article interesting, then you can explore “Mastering Kibana 6.0”, “Kibana 7 Quick Start Guide”, “Learning Kibana 7”, and “Elasticsearch 7 Quick Start Guide” books to get more insight about Elastic Stack and how we can create dashboards for key performance indicators using Kibana.

Originally published at https://bqstack.com.

--

--

--

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Recommended from Medium

How to handle Guzzle exception and get Http Body?

Day 1: Committing to 30 days of Google Cloud Challenge

LESS / SASS / SCSS Are Junk Too

Hybrid and Multi-Cloud Overlay — Part 5— Challenges — AWS, Azure, GCP, OCI and Alicloud

🔥 Upcoming Voice AMA with Pinksale!

Diameter of N-Ary Tree

How I won $400 just by coding and playing games

Juce for Beginners, 2020 edition

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Anurag Srivastava

Anurag Srivastava

Author of “Mastering Kibana6.x”, “Kibana7 Quick Start Guide”, “Learning Kibana7”, &“Elasticsearch7 Quick Start Guide” books & AWS Certified Solutions Architect.

More from Medium

Setting up Hadoop with Docker and using MapReduce framework

Fetching Data from Twitter to MongoDB

Discussion on some concealed issues faced in end-to-end data management Part I

Kinesis Data Stream Working and Architecture — Part2