Building Search API with ElasticSearch — Part 04: Search Query

4 min readJul 7, 2022

In this post I will present a not so simple search query, it will be composed of some criteria and boost to increase the relevance of some fields. The proposal is not to write a perfect query, because this query does not exist. We must meet the needs of business rules and return search results relevant to the user’s search context.

First of all, working with search relevance is a complicated task, results that meet a search context are not always an idea for another user. In this project I’m defining what I want to be relevant and defining criteria that our query has to meet.

In a search in our movie index, I believe that titles are the most common searches and that’s why I’m going to define a “match” criterion on the title and this clause will have greater relevance over the others. I’ll do this by applying a boost to the “match” clause. This clause will require that the informed term or terms exist in the title.

This criterion will help in my accuracy because I will return documents with the exact search terms.

The next criterion will be to search for the term in several fields, in this way it would have a greater document retrieval. Why that? Sometimes the user does not type exactly the title of the movie he wants or the name of the director or actor.

At this point I intend to return the most relevant documents to the terms informed, so we can say that we will have a significant recovery rate.
In the recovery criterion I will use the “multi-match” and search on the fields “title”, “actors”, “director” and “description. In this clause I will also use the boost for each field.

Java Code

Writing the code after testing the new query is a very simple task. A Bool Query was used together with MatchQuery and MultiMatchQuery, our complete query can be visualized in the ServiceCore class.

An important point is the boost values used, I left the highest value in the title because according to our rule it is where the relevance should be greater.
It is important to know that there is no standard formula to define field boosts. You must perform tests and understand the results and thus assume that the values are correct and have resulted in the expected results.

Now let’s start our tests and check the search results.

Running tests

Our first test will be the search for the movie “Lion”, there are several movies with the term “lion” but we want the most relevant to be ranked in the first position since the match is exact in the title.

curl --location --request GET 'http://localhost:8080/api/movies/search?size=10&text=Lion'

Obtemos 3 resultados na seguinte ordem:

Lion
The Lion King
The Lion in Winter

It happened as we wished, we have the movie “Lion” and we also recovered other movies related to the term.

Now let’s search for “lion king”. The result should now have the movie “The Lion King” in the first position and recover the others.

curl --location --request GET 'http://localhost:8080/api/movies/search?size=10&text=lion king'

The list is as expected, “The Lion King” is at the top. However, we retrieve a larger amount of documents that may not be relevant.

This is where the analysis work comes in to understand if this amount of documents returned are relevant to the user and from there, new tests will be needed to improve the query, mapping and boost in fields.

Writing a query that satisfies the user with relevant results is a very complicated task and therefore we cannot say that there is a silver bullet. Anyway, I recommend a book, Relevant Search, which was very important for my understanding of the relevance of results.

In the next post we will continue to add more features to our API.

References

Elasticsearch Java API Client

Edit description

www.elastic.co

Multi-match query | Elasticsearch Guide [8.3] | Elastic

Elasticsearch Guide [8.3] " "Full text queries " Multi-match query