Deploying the ELK stack on Amazon ECS, Part 5: Kibana
This is Part 5 of a multi-part series on how to deploy a containerized, production-ready ELK stack in Amazon’s EC2 container service.
Please refer to Part 1, Part 2, Part 3, Part 4 for the previous tutorials.
At this point, you should have your ECS cluster deployed, ElasticSearch container configured and running, with the Application ELBs routing traffic across all nodes. In addition, logstash should be deployed and running, accepting TCP traffic on port 5000.
With that, let’s take a look at how to configure and deploy Kibana.
Kibana Docker container deployment is the easiest step in this entire recipe! Fortunately for us, Elastic exposed nearly all the config options that matter via environment variables — an outstanding application of the Heroku’s 12 factor app principles.
What this means in practice is that we do not need to bake our custom Kibana container, we can just use Elastic’s and configure it to suit our needs.
Take a look at a working Kibana task definition:
Items of note:
- ELASTICSEARCH_URL: this points to the Route53 entry created as part of the previous section on ElasticSearch deployment. This URL points to the Application Load Balancer that sits in front of our ElasticSearch Docker containers.
- Kibana image: We can use Elastic’s standard container since we are not customizing it in any way. The URL is docker.elastic.co/kibana/kibana:5.5.0
- Soft mem limit: 1024MB, adjust if you need more
- Port mappings: Host port 0 (meaning, let ECS assign the host port) and 5601 for the port inside the Docker container.
Next, let’s configure the Application Load Balancer to route traffic between multiple Kibana containers.
Remember, we don’t need individual Application ELBs for every service! Therefore, we will reuse the ELB we created previously. All we need to do here is create a new target group.
Use /app/kibana for the health check:
NOTE: If you create a new Route53 entry to point to the same ELB, then you can refer to this easy to remember entry in your ELB rule table:
ecs-platfo-kibana is what I named the target group.
Finally, take a look at a working service definition for Kibana:
Set your desired count to whatever you want. A minimum of 2 is suggested for a highly available, load-balanced deployment.
That’s it!
When you navigate to your kibana URL, you should see an empty Kibana UI, asking you for a default index. If you have not submitted any entries, you will not have any indices.
Next, we’ll configure awslogs to send our docker logs to ElasticSearch.
That’s Part 6!
