Mastering Elastic DSL: Essential Queries Every Elastic Developer Should Master

In Elasticsearch, Query DSL is a powerful way to define and execute complex queries against your data. It allows you to specify the criteria and conditions for your search, including filters, aggregations, and sorting. Query DSL provides a structured and expressive means of querying Elasticsearch indexes and is typically used in JSON format.

Here’s a basic example of an Elasticsearch query written in Query DSL JSON format:

GET Some_Index_Name/_search
{
  "query": {
    "match": {
      "title": "Elasticsearch"
    }
  }
}

We will start form the most basic then slowly move to the complex queries.

To retrieve all documents from an Elasticsearch index using Query DSL, we use “match_all” query. It is similar to select * from table in MySQL.

Here’s an example of how to do it:

{
  "query": {
    "match_all": {}
  }
}

In this Query DSL JSON snippet:

1. "query" is the key indicating that we are defining a query.
2. "match_all" is the query type used to match all documents.

You can specify the index you want to query by including the index name in your Elasticsearch request. For example, if you want to query an index named “my_index,” your request might look like this:

GET MyIndex/_search
{
  "query": {
    "match_all": {}
  }
}

Note : If you execute this query you will notice that you got only 10 documents but not all the documents. Because Elastic only returns 10 docuements by default.

Next Question is, How to get more than 10 documents using match all query ?

To retrieve a specific number of documents (in this case, 100 documents) from Elasticsearch using a match_all query, you can use the “size” parameter to limit the number of results returned. Here’s an example Query DSL JSON snippet to achieve this:

{
  "size": 100,
  "query": {
    "match_all": {}
  }
}

In this query:

1. "size": 100 specifies that you want to retrieve a maximum of 100 documents.
2. "query" is the key indicating that we are defining a query.
3. "match_all" is the query type used to match all documents.

Now, Third thing you will notice that, Elastic has returned all the fields in the response.

For Example :-

{
  "took": 10,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 100,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "my_index",
        "_type": "_doc",
        "_id": "1",
        "_score": 1.0,
        "_source": {
          "field1": "value1",
          "field2": "value2"
        }
      },
      {
        "_index": "my_index",
        "_type": "_doc",
        "_id": "2",
        "_score": 1.0,
        "_source": {
          "field1": "value3",
          "field2": "value4"
        }
      },
      // ... more documents ...
    ]
  }
}

Here’s an explanation of the key elements in this response:

• "took": Indicates how long the query took to execute in milliseconds.
• "_shards": Provides information about the shard-level execution of the query.
• "hits": Contains information about the documents that matched the query.
• "total": Shows the total number of documents that matched the query. In this example, there are 100 matching documents.
• "max_score": The maximum relevance score among the matching documents.
• "hits": An array of individual document hits, each containing:
• "_index": The index where the document is stored.
• "_type": The type of the document (if using types; note that types are deprecated in recent versions of Elasticsearch).
• "_id": The document's unique identifier.
• "_score": The relevance score of the document (in a match_all query, all documents have the same score of 1.0).
• "_source": The actual document data as stored in Elasticsearch, including all the fields and their values.

In the next query, we will understand how to get selective fields only.

To retrieve selective fields (also known as source filtering) from Elasticsearch using Query DSL, you can use the “_source” parameter in your query to specify which fields you want to include or exclude from the search results. Here’s how you can do it:

Suppose you have an index named “my_index,” and you want to retrieve only the “field1” and “field2” from the documents. Your Query DSL JSON request might look like this:

{
  "_source": ["field1", "field2"],
  "query": {
    "match_all": {}
  }
}

If you want to exclude specific fields and retrieve all other fields, you can use the “exclude” option within the “_source” parameter like this:

{
  "_source": {
    "excludes": ["field_to_exclude"]
  },
  "query": {
    "match_all": {}
  }
}

In this above case, the response will include all fields except the one specified in the “excludes” array.

A “match” query in Elasticsearch Query DSL is used to perform a full-text search on one or more fields within the documents in an index. Here’s how you can write a simple “match” query:

{
  "query": {
    "match": {
      "field_name": "Text1 Text2"
    }
  }
}

This basic “match” query will search for documents where the specified field contains the specified text. Elasticsearch will return documents that match the search criteria, and it will also calculate a relevance score to rank the results based on how well they match the query.

You can also customize the “match” query with various options and settings, such as specifying a different operator, boosting certain terms, or using a different analyzer for the search text. These options allow you to fine-tune the behaviour of your full-text search query in Elasticsearch.

Note : Above query will look for Text1 OR Text2 in all the documents and if either of these keywords are present it will return them in response.

If you would like to get only documents which should contain both the keywords. Then you will have to use AND operator and need to modify the query a little bit.

GET My_Index/_search
{
  "query": {
      "match": {
        "content": {
          "query": "text1 text2",
          "operator": "and"
        }
    }
  }
}

Note : Above query will look for both the keywords in the documents but not together.

Suppose, you want that Elastic should return only those documents if both the keywords come together in the given order as above or you can say if they match a certain phrase.

For example, if you have a field called "description" and you want to find documents containing the exact phrase "This blog is very informative," your "match_phrase" query would look like this:

{
  "query": {
    "match_phrase": {
      "description": "This blog is very informative,"
    }
  }
}

Some time, you need to sort your result with some field. To sort documents by a field like “publish_date” in Elasticsearch Query DSL, you can use the “sort” parameter within a “bool” query. Here’s an example of how to write a Query DSL query to sort documents by the “publish_date” field in descending order (from the most recent to the oldest):

{
  "sort": [
    {
      "publish_date": {
        "order": "desc"
      }
    }
  ]
}

Today, we’ve explored numerous practical applications of DSL Queries in real-life scenarios. Here are a few examples:

Understanding the usage of the match_all query.

Retrieving multiple documents efficiently using the match_all query.

Selectively extracting specific fields in the response.

Filtering results with and without operators.

Leveraging the match_phrase query for precise searches.

Implementing result sorting based on specific field criteria.

In my upcoming blog post, we will delve into more useful query types.

If you have any questions about today’s topics or the queries I’ve covered, please don’t hesitate to reach out. Until then, stay well, keep learning and take care! 😊