Building Search API with ElasticSearch — Part 02: Pagination

3 min readJul 6, 2022

In our second part of building the search API we are going to add Pagination.

Using pagination is very important because we don’t want to traffic large amounts of data and not request the entire database in a single request. By default size of ElasticSearch returns 10 documents but our application the “size” can be between 10 and 20 documents.

The page strategy used will be SearchAfter. SearchAfter is a strategy that allows you to search for the next page from the last hit returned on the previous page.

In our implementation we will use searchAfter passing two parameters as a starting point for the next page. The fields will be the “code” and “_score”. The score is the relevance value generated by the BM25 algorithm. From these fields searchAfter will know that from these values it will search for new documents.

In our SearchDTO we will add the properties “searchAfterScore” and “searchAfterCode “ that respectively represent the values of score and code of the movie to be informed in the request.

In the “getAll” method of the SearchCoreService class we will add the searchAfter to the query. Note that the query receives a list of strings, so it was necessary to convert the score and code into a list as seen here.

See how our “getAll” method looked after adding the searchAfter here.

The next phase is to return the list of documents informing “score” and “code” in each one, so that the front will know which value to send in the next request and receive the new page.

This change will add a new property in MovieCatalogDTO, the property will be named “searchAfter”, see here as implemented.

Now that we have the implementation finished, let’s run our tests.

Our first search will be this:

curl --location --request GET 'http://localhost:8080/api/movies/all?size=10&text'

We didn’t inform the searchAfter, so we’ll start from the beginning.

The movie list was returned and notice that the “searchAfter” property is filled with the score and code values.

To search for the next page, just pass the searchAfterCode = 10 and searchAfterScore = 1 in the next request. The next documents will be from position 11 to 20.

curl --location --request GET 'http://localhost:8080/api/movies/all?size=10&searchAfterScore=1&searchAfterCode=10&text'

See that the results have been paginated the way we expected.

With that we end the pagination part with searchAfter, in the next post we will implement the autocomplete solution.

Source Code
https://github.com/andreluiz1987/search-store

Building Search API with ElasticSearch — Part 02: Pagination

References

Paginate search results | Elasticsearch Guide [8.3] | Elastic

By default, searches return the top 10 matching hits. To page through a larger set of results, you can use the search…

Written by André Coelho