How to simplify Docker container log analysis with Elastic Stack

Background Image Courtesy — | Created via
  • Imagine you have tens, hundreds, or even thousands of containers generating logs — SSH-ing in to all those servers and extracting logs won’t work well.
  • Also containers are immutable and ephemeral, which means they have a shorter life span. So once your containers are gone and replaced with new containers, all of your application logs related to old containers are gone.
  • Beats is the new member which made the ELK Stack known as Elastic Stack. Beats are light weight log data shippers which can push logs to the ELK Stack. For this post I will be using Filebeats, a member of the Beats family, which offers a lightweight way to collect and forward and centralize logs and files.
  • Logstash is a component which aggregates, modifies, and transfers logs from multiple input locations into Elasticsearch.
  • Elasticsearch is a distributed, JSON-based search and analytics engine that stores and indexes data (log entries in this case) in a scalable and manageable way.
  • Kibana is an enriched UI to analyze and easily access data in Elasticsearch.
High Level Architecture — Instance 1 [Left] | Instance 2 [Right]

Configuring Filebeat

docker run -v '/var/lib/docker/containers:/usr/share/dockerlogs/data:ro' -v '/var/run/docker.sock:/var/run/docker.sock' --name filebeat ${YOUR_FILEBEAT_DOCKER_IMAGE}:latest

Configuring the ELK Stack

Kibana Dashboard
Filebeat Index Patten Configuration on Kibana Dashboard
Discover Docker container application logs along with the Docker host metadata in Kibana Dashboard



We’ve moved to and publish tons of tutorials each week. See you there.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store