Building Search API with ElasticSearch— Part 01: Preparing the environment

3 min readJul 6, 2022

As stated in previous posts we are going to build a search API using all the features presented so far and we will add observability through Elastic APM Server.

Our API will be built with Java 11, SpringBoot and the current Elastic client library: Java API Client in addition to the Agent APM java.

The index will be composed of movie data, we will use a dataset that will need to be manipulated before indexing, but I will leave the bulk script of the documents available to speed up the construction of the environment.

Let’s add the following functions to the API:

Autocomplete (Completion Suggester)
Pagination
Facets (Aggregations)
Traditional search
“Did you mean”

At the end we will add the APM Java Agent to monitor the API and we will visualize all the metrics collected in Kibana.

Last but not least, a FrontEnd application will also be created for API requests.

Provisioning Services

We will use docker compose to provision Elastic services like ElasticSearch, Kibana and APM Server.
Below we can see the docker compose file with all the services declared:

version: '2.2'services:elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.1
    container_name: elasticsearch-7.17.1
    environment:
      - node.name=elasticsearch
      - discovery.seed_hosts=elasticsearch
      - cluster.initial_master_nodes=elasticsearch
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200kibana:
    image: docker.elastic.co/kibana/kibana:7.17.1
    container_name: kibana-7.17.1
    environment:
      ELASTICSEARCH_URL: "http://elasticsearch:9200"
    ports:
      - 5601:5601
    depends_on:
      - elasticsearchapm-server:
    image: docker.elastic.co/apm/apm-server:7.17.1
    cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]
    cap_drop: ["ALL"]
    ports:
    - 8200:8200
    command: >
       apm-server -e
         -E apm-server.rum.enabled=true
         -E setup.kibana.host=kibana:5601
         -E setup.template.settings.index.number_of_replicas=0
         -E apm-server.kibana.enabled=true
         -E apm-server.kibana.host=kibana:5601
         -E output.elasticsearch.hosts=["elasticsearch:9200"]
    healthcheck:
      interval: 10s
      retries: 12
      test: curl --write-out 'HTTP %{http_code}' --fail --silent --output /dev/null http://localhost:8200/volumes:
  esdata1:
    driver: local

Now that we have the ElasticSearch created we must create our index and index the documents. I’m going to use a movie dataset and I’m going to create the index with the name “idx_movies”.

The index mapping can be found here and the documents for indexing here.

Java API + Spring

The API can be downloaded by clicking on this link. This first version of the API has only one method that fetches all movies using pagination.
Some validations were created but feel free to improve the code because this version I’m presenting is just for demonstration purposes.

After downloading the source code and its dependencies, it is possible to test our first endpoint.

All our search queries were in the “core” package. In this version we will have only one method that will contain a “match_all” and will return a list of movies.

I’m using POSTMAN to make the requests, import curl below to make the call.

curl --location --request GET 'http://localhost:8080/api/movies/all?size=10&from=1' \
--header 'Content-Type: application/json'

If it occurs correctly, the list of movies will be returned.

This is the first part of building our API. I hope you were able to upload the services and run the API by running the first endpoint.

Source Code
https://github.com/andreluiz1987/search-store

In the next post we will add more functionality and robustness to the API.

References

APM Java Agent Reference

Edit description

www.elastic.co

Elasticsearch Java API Client

Edit description

www.elastic.co

Building Search API with ElasticSearch— Part 01: Preparing the environment

Provisioning Services

Java API + Spring

References

APM Java Agent Reference

Edit description

Elasticsearch Java API Client

Edit description

Install Kibana with Docker | Kibana Guide [8.3] | Elastic

Docker images for Kibana are available from the Elastic Docker registry. The base image is ubuntu:20.04. A list of all…

Written by André Coelho