To Play Around ELK in the Docker Playground

mingderwang
Kubernetes Playground
4 min readOct 3, 2017

If you don’t know how to use Docker Playground, please read my previous log, “Setup a Docker Swarm in Docker Playground in X mins

You only need docker-compose (default installed on each node) for ELK, you don’t need to install Docker Swarm. Create a new instance and then simply run the following commands to start an ELK working environment.

wget https://gist.githubusercontent.com/nomkhonwaan/41c352721aa2c327abb50825e2c90369/raw/a597ffd994c29bb0fef09bbe9a9e8d9509150af6/docker-compose.ymldocker-compose up
// or
docker-compose up -d

Click on the NodePort 5601 showed above.

then, you can see a kibana 5.5.2 home page with an elasticsearch running in the background.

Take a look at the k8s yaml file for ELK in this example, which we wget from https://gist.githubusercontent.com/nomkhonwaan/41c352721aa2c327abb50825e2c90369/raw/a597ffd994c29bb0fef09bbe9a9e8d9509150af6/docker-compose.yml and we can do some modification as folllows;

version: '2'
services:
elasticsearch:
image: elasticsearch:latest
environment:
ES_JAVA_OPTS: -Xms4g -Xmx4g
logstash:
image: logstash:latest
volumes:
- $PWD/config-dir:/config-dir
- $PWD/logs:/logs
depends_on:
- elasticsearch
links:
- elasticsearch
command: logstash -f /config-dir/logstash.conf
ports:
- 5000:5000
kibana:
image: kibana:latest
ports:
- 5601:5601
depends_on:
- elasticsearch
links:
- elasticsearch

Because we have 8GB memory per node in the k8s or docker playground, so that, we can change ES_JAVA_OPTS: to -Xms4g -Xmx4g for the best practice.

And also add ports: 5000:5000 for logstash in order to stream syslog sample logs into logstash.

And we also know $PWD/config-der is the logstash config path we can use as the /etc/logstash/conf.d directory we can use as in the normal ELK installation.

So that, you can create a sample logstash config file in your $PWD/config-der/logstash.conf folder, such as logstash-syslog.conf as follow;

cd
mkdir -p $PWD/config-dir/logstash.conf

cat $PWD/config-dir/logstash.conf/logstash-syslog.conf

input {
tcp {
port => 5000
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch { hosts => ["elasticsearch:9200"] }
stdout { codec => rubydebug }
}

After you change logstash config file, you need to restart ELK by

docker-compose restart

It should re-run logstash with your config files under $PWD/config-dir/logstash.conf/ directory.

Now, you can try to stream some syslog to logstash on port 5000 from local or another instance (node IP: 10.0.9.4 for example). Such as

telnet 10.0.9.3 5000
// or
cat <file> | nc -w 1 -v localhost 5000

followed by the syslog samples bellow.

Dec 23 12:11:43 louis postfix/smtpd[31499]: connect from unknown[95.75.93.154]
Dec 23 14:42:56 louis named[16000]: client 199.48.164.7#64817: query (cache) 'amsterdamboothuren.com/MX/IN' denied
Dec 23 14:30:01 louis CRON[619]: (www-data) CMD (php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log)
Dec 22 18:28:06 louis rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="2253" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.

Where, 10.0.9.3 is the node run ELK, and port 5000 is logstash input port.

and check the logs of logstash container on the ELK node (10,0.9.3) as follow

docker psCONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS                    NAMES
69524ebd6ce8 logstash:latest "/docker-entrypoin..." 10 minutes ago Up 10 minutes 0.0.0.0:5000->5000/tcp root_logstash_1
b54847eb3c9f kibana:latest "/docker-entrypoin..." About an hour ago Up 12 minutes 0.0.0.0:5601->5601/tcp root_kibana_1
0930b302d5a7 elasticsearch:latest "/docker-entrypoin..." About an hour ago Up 12 minutes 9200/tcp, 9300/tcp root_elasticsearch_1

where 69524ebd6ce8 (695) is the container ID for logstash, so we can use docker logs -f 695 to see the parsed logs as follows;

docker logs 695

Now you can use kibana to see the logs too.

Further reading for Logstash Configuration you can refer to https://www.elastic.co/guide/en/logstash/current/config-examples.html with sample examples in this blog.

--

--