Setting up a Load-balanced Logstash behind and AWS ELB

Logstash running on EC2 instances behind an AWS ELB.

Step 1: Instance Launch Template and Autoscaling Group

To create an autoscaling group we can start off by creating one of two things:

  1. Launch Configuration
  2. Launch Templates (newer)
Launch Template creation page
Adding User data to Launch Template
#!/bin/bash
cd /tmp/
sudo yum -y install java
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.4.rpm
rpm -vi logstash-6.5.4.rpm
access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/my-logstash-role | python -c 'import sys, json; print json.load(sys.stdin)["AccessKeyId"]'`
secret_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/my-logstash-role | python -c 'import sys, json; print json.load(sys.stdin)["SecretAccessKey"]'`
session_token=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/my-logstash-role | python -c 'import sys, json; print json.load(sys.stdin)["Token"]'`
mkdir /root/.aws/
echo "[default]
aws_access_key_id=$access_key
aws_secret_access_key=$secret_key
aws_session_token=$session_token
region=us-west-2" > /root/.aws/credentials
aws secretsmanager get-secret-value --secret-id My_Logstash_Configuration | python -c 'import sys, json; print json.load(sys.stdin)["SecretString"]' > /etc/logstash/conf.d/log_config1.conf
/usr/share/logstash/bin/logstash-plugin install logstash-filter-geoip
sudo mkdir -p /usr/share/logstash/geoip/
sudo curl https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz | tar -xz -C /tmp/
sudo curl https://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz | tar -xz -C /tmp/
sudo cp /tmp/GeoLite2-City*/GeoLite2-City.mmdb /usr/share/logstash/geoip/
sudo cp /tmp/GeoLite2-ASN*/GeoLite2-ASN.mmdb /usr/share/logstash/geoip/
sudo chown logstash:logstash /usr/share/logstash/geoip/
echo 'echo "path.data: /var/lib/logstash
queue.type: persisted
path.queue: /tmp/logstash/queue
queue.max_bytes: 2gb
path.logs: /var/log/logstash" > /etc/logstash/logstash.yml' | sudo -s
sudo service logstash start
169.254.169.254/latest/meta-data/iam/security-credentials/<role_name>

Step 2: ELB Configuration

You would have to create a Network Load Balancer from the available load balancers if you want to listen for beats (filebeat, heartbeat etc.).

Creating a target group for the network load balancer

Step 3: The Filebeats Configuration

The logstash configuration remains the same as it would in a single node logstash deployment. But the thing that changes is the configuration of the beats that connect to logstash.

output.logstash:
hosts: ["blah_blah_blah.elb.us-west-2.amazonaws.com:5044"]
ttl: 120

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store