Building Search API with ElasticSearch— Part 01: Preparing the environment

André Coelho
3 min readJul 6, 2022

--

As stated in previous posts we are going to build a search API using all the features presented so far and we will add observability through Elastic APM Server.

Our API will be built with Java 11, SpringBoot and the current Elastic client library: Java API Client in addition to the Agent APM java.

The index will be composed of movie data, we will use a dataset that will need to be manipulated before indexing, but I will leave the bulk script of the documents available to speed up the construction of the environment.

Let’s add the following functions to the API:

  • Autocomplete (Completion Suggester)
  • Pagination
  • Facets (Aggregations)
  • Traditional search
  • “Did you mean”

At the end we will add the APM Java Agent to monitor the API and we will visualize all the metrics collected in Kibana.

Last but not least, a FrontEnd application will also be created for API requests.

Provisioning Services

We will use docker compose to provision Elastic services like ElasticSearch, Kibana and APM Server.
Below we can see the docker compose file with all the services declared:

version: '2.2'services:elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.1
container_name: elasticsearch-7.17.1
environment:
- node.name=elasticsearch
- discovery.seed_hosts=elasticsearch
- cluster.initial_master_nodes=elasticsearch
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
kibana:
image: docker.elastic.co/kibana/kibana:7.17.1
container_name: kibana-7.17.1
environment:
ELASTICSEARCH_URL: "http://elasticsearch:9200"
ports:
- 5601:5601
depends_on:
- elasticsearch
apm-server:
image: docker.elastic.co/apm/apm-server:7.17.1
cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]
cap_drop: ["ALL"]
ports:
- 8200:8200
command: >
apm-server -e
-E apm-server.rum.enabled=true
-E setup.kibana.host=kibana:5601
-E setup.template.settings.index.number_of_replicas=0
-E apm-server.kibana.enabled=true
-E apm-server.kibana.host=kibana:5601
-E output.elasticsearch.hosts=["elasticsearch:9200"]
healthcheck:
interval: 10s
retries: 12
test: curl --write-out 'HTTP %{http_code}' --fail --silent --output /dev/null http://localhost:8200/
volumes:
esdata1:
driver: local

Now that we have the ElasticSearch created we must create our index and index the documents. I’m going to use a movie dataset and I’m going to create the index with the name “idx_movies”.

The index mapping can be found here and the documents for indexing here.

Java API + Spring

The API can be downloaded by clicking on this link. This first version of the API has only one method that fetches all movies using pagination.
Some validations were created but feel free to improve the code because this version I’m presenting is just for demonstration purposes.

After downloading the source code and its dependencies, it is possible to test our first endpoint.

All our search queries were in the “core” package. In this version we will have only one method that will contain a “match_all” and will return a list of movies.

I’m using POSTMAN to make the requests, import curl below to make the call.

curl --location --request GET 'http://localhost:8080/api/movies/all?size=10&from=1' \
--header 'Content-Type: application/json'

If it occurs correctly, the list of movies will be returned.

Response Movies Match All

This is the first part of building our API. I hope you were able to upload the services and run the API by running the first endpoint.

Source Code
https://github.com/andreluiz1987/search-store

In the next post we will add more functionality and robustness to the API.

Building Search API with ElasticSearch — Part 02: Pagination
Building Search API with ElasticSearch — Part 03: Autocomplete
Building Search API with ElasticSearch — Part 04: Search Query
Building Search API with ElasticSearch — Part 05: Simple Facets
Building Search API with ElasticSearch — Part 06: Filters
Building Search API with ElasticSearch — Part 07: “Did you mean”
Building Search API with ElasticSearch — Part 08: Sort results
Building Search API with ElasticSearch — Part 09: Recommendation with More Like This Query
Building Search API with ElasticSearch — Part 10: Front-End
Building Search API with ElasticSearch — Part 11: Elastic APM

--

--

André Coelho

Developer of web and mobile systems. Enthusiast in the area of ​​automation and electronics and I have hobbie music.