Elasticsearch : Common Problems & Possible Fixes

I have encountered some problems while using Elasticsearch . In this article, I will share how I fixed these issues. Hope you’ll find it useful for troubleshooting.

Problem 1 — Index mapping cannot be edited

Explanation

Index mapping should be defined during index creation. If mapping is not defined, Elasticsearch will iterate over each indexed field of the JSON document, estimates and creates its own respective mapping. Although it seems ideal but it might not always accurate for project use. It is possible for Elasticsearch to pick wrong field type mapping.

Fix

Remove the index and create a new index with new mapping.

Problem 2 — Flood stage disk watermark exceeded

Explanation

Elasticsearch decides whether to allocate new shards, relocate shards away or block all index write operations on a data node based on available disk space.

cluster.routing.allocation.disk.watermark — There are three thresholds: low, high, and flood_stage. These can be changed dynamically, accepting absolute values as well as percentage values. Threshold can be specified both as percentage and byte values, but percentage is more flexible and easier to maintain as different nodes might have different disk sizes.

Fix

Change the settings values to a higher threshold by dynamically updating the settings using update cluster API:

PUT _cluster/settings

Update Elasticsearch Index Settings

Other possible fixes

1. Delete unused indices.
2. Attach external disk or increase the disk used by the data node.
3. Manually move shards away from the node using cluster reroute API.
4. Reduce replicas count to 1 (if replicas > 1).
5. Add new data nodes.

Problem 3 — Wrong Field Type for Terms Aggregation

Explanation

To run a terms aggregation, field type must be either keyword, numeric, ip , boolean or binary data. Text is not allowed for terms aggregation by default.

Fix

1. Add a new field using keyword as field type OR
2. Enable fielddata on existing field with text as field type. It will create buckets for the field’s analyzed terms. However, it can significantly increase memory usage for this approach.

If you are still stuck with your problems, feel free to access the official guide of elasticsearch.

Elasticsearch Guide