Igor, Thank you very much for putting together all the instructions. I was able to setup the ES cluster on AWS. Only one thing that I had to do differently is to overide the new Launch Configuration Name directly on the Cloud Formation template as its not taking effect on ASG. If I do it on ASG, the Cloud Formation is overriding it. Anyway, that is the only difference.
In your proposed setup, every ES node, by default, acting as data, master and ingest. I m trying to configure differently by separating them as individual nodes — ex : 3 data ONLY, 2 master ONLY and one ingest ONLY node. To set that up, I created 3 different Task Defintions and corresponding ECS Service for each TD. I used the environment variables to control the node behaviour using these threee variables IS_MASTER,IS_DATA and IS_INGEST. I started the Service with Data Nodes and cluster came up with 3 nodes perfectly fine. Then I started the master Service but thats not able to join the cluster. For Data Node Service, I provided explict port mappings 9200 →9200 and 9300 → 9300 and for master I configured dynamic port mapping to avoid the port conflicts 0 → 9300 as both master, data and ingest run on same machine. Not sure if that is the culprit. Anyway, Have u ever come across this setup. can you shed some light what might be happening?
Error : “failed to send join request to master reason [RemoteTransportException[[jVb1EHy][172.17.0.2:9300][internal:discovery/zen/join]]; nested:“