Getting started with Metricbeat

How to monitor system metrics data using Metricbeat and elasticsearch.

Photo by Campaign Creators on Unsplash

IT systems can be monitored for various attributes such as performance, reliability, availability etc. Metrics is the measurement of behaviour and usage of system resources that can be collected and monitored from the system.

In today’s day and age of highly complex IT systems and environments, monitoring of system metrics helps increase the availability and reliability of the system and allows IT teams to quickly respond to any fault if they occur.

ELK stack became elastic stack with the addition of beats, a family of light-weight shippers used for different use cases and data sets. In the previous articles, I gave an overview of the elastic stack and installed an elastic search on Linux. In this article, I’ll focus on Metricbeat.

Metricbeat is a lightweight agent that can be installed on target servers to periodically collect metric data from your target servers, this could be operating system metrics such as CPU or memory or data related to services running on the server. It can also be used to monitor other beats and ELK stack itself. It is built on top of libbeat — a GO framework.

It can be configured to send output to elasticsearch directly or to logstash if you need to transform the data.

Lab Setup

In this article, I’ll set up a single-node elastic search cluster(refer to this article) and two apache webservers. I have used the GCP platform to build my test lab since it offers $300 USD free trial credit but you can do it on your servers or any other public cloud platform as well.

We’ll use metricbeat’s internal collection method to collect monitoring data about our elasticsearch cluster.

We’ll also setup metricbeat to collect metric data from our webservers and send it to elasticsearch.

Metricbeat Installation

Note: You’ll need an existing elasticsearch cluster to store metrics data and Kibana to visualise this data. Please read this article to help set up a single-node elasticsearch instance and Kibana.

We’ll use the APT repository method to install Metricbeat.

Installation and configuration of Metricbeat on ELK Server

1. Enable Monitoring on Elasticsearch Cluster

Before we can install metricbeat to monitor our ELK cluster, we first need to enable monitoring in our elasticsearch.yml file

cd /etc/elasticsearch
sudo nano /etc/elasticsearch/elasticsearch.yml
xpack.monitoring.collection.enabled: true
xpack.monitoring.elasticsearch.collection.enabled: true

2. Restart Elasclearticsearch service

sudo systemctl restart elasticsearch

3. Install Metricbeat

Note: Steps 3.1–3.3 are not necessary if you have previously set up your APT repository during elasticsearch deployment

3.1. Download and install public signing key

wget -qO — https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

3.2. Install apt-transport-https package

sudo apt-get install apt-transport-https -y

3.3. Save directory definitions

echo “deb https://artifacts.elastic.co/packages/7.x/apt stable main” | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

3.4. Install Metricbeat

sudo apt-get update && sudo apt-get install metricbeat

4. Configure Metribeat

Metricbeat configuration can be found in /etc/metricbeat/metricbeat.yml file. Please be careful while configuring this file since YAML is syntax sensitive. Please feel free to read this article to learn a bit more about YAML.

sudo su
nano /etc/metricbeatmetricbeat.yml
### live monitoring section
reload.enabled: true
reload.period: 10s
### name and tag
name: elk-metrics
Tags: demo-elk
env: lab
### enable kibana dashboards
Setup.dashboards.enabled: true
host: <<ip_address_of_elastic_server>>:5601
### output configuration
output.elasticsearch:
# Array of hosts to connect to.
hosts: [“<<ip_address_of_elastic_server>>:9200”]
# Protocol — either `http` (default) or `https`.
protocol: “https”
username: “elastic” #### elasticsuperuser
password: “password”. #### elastic users password
## logging
logging.level: debug
logging.selectors: [“*”]
logging.to_files: true
logging.files:
path: /var/log/metricbeat
name: metricbeat
keepfiles: 7
permissions: 0644
## enable interal collection of metricbeat
monitoring.enabled: true
monitoring.elasticsearch

one configuration I’d like to point out in the metricbeat.yml configuration file is live reload

reload.enabled: true
reload.period: 10s

this config allows the user to make changes to module configurations files without the need to restart metricbeat service. Metricbeat checks if there have been any changes made to the module configuration filed based on the timer set in reload.period setting.

5. Start Metricbeat

sudo systemctl start metricbeat

6. Modules and metric sets

Metricbeat does its magic by using modules and metric sets. Let's try and understand these:

Let's say you want to monitor one of your Linux servers for its CPU, load, memory etc. what you’ll need to do is after you install metric beat on the VM you need to enable a module called system which provides system monitoring information. The system module allows you to monitor a lot of things as listed below

metricsets:
 — cpu # CPU usage
 — load # CPU load averages
 — memory # Memory usage
 — network # Network IO
 — process # Per process metrics
 — process_summary # Process summary
 — uptime # System Uptime
 — socket_summary # Socket summary
 #- core # Per CPU core usage
 #- diskio # Disk IO
 #- filesystem # File system usage for each mountpoint
 #- fsstat # File system summary metrics
 #- raid # Raid
 #- socket # Sockets and connection info (linux only)
 #- service # systemd service information

Each of these items individually is called a metric set. You can either choose to enable all of these or only the ones that you are interested in.

6.1 Configure Modules for ELK servers

For our elasticsearch cluster, we’ll configure

• elasticsearch-xpack
• Kibana-xpack

6.1 Enable elasticsearch-xpack

First, we’ll need to enable the module using the following

sudo metricbeat modules enable elasticsearch-xpack

After the module has been enabled, it can be configured using /etc/metricbeat/modules.d/elasticsearch-xpack.yml file

nano /etc/metricbeat/modules.d/elasticsearch-xpack.yml
- module: elasticsearch
    xpack.enabled: true
    period: 10s
    hosts: ["http://localhost:9200"] 
    #scope: node 
    username: "remote_monitoring_user"
    password: "remote_monitoring_user_password"
    #ssl.enabled: true
    #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
    #ssl.certificate: "/etc/pki/client/cert.pem"
    #ssl.key: "/etc/pki/client/cert.key"
    #ssl.verification_mode: "full"
    xpack.enabled: true

6.2 Enable Kibana-xpack

As we’ve done earlier, first we’ll need to enable Kibana-xpack module

sudo metricbeat modules enable kibana-xpack

After the module has been enabled, it can be configured using /etc/metricbeat/modules.d/kibana-xpack.yml file

nano /etc/metricbeat/modules.d/kibana-xpack.yml
- module: kibana
  metricsets:
    - stats
  period: 10s
  hosts: ["localhost:5601"]
  #basepath: ""
  #username: "remote_monitoring_user"
  #password: "remote_monitoring_user_password"
  xpack.enabled: true

Installation and configuration of Metricbeat on WebServer

Note: Repeat steps 1–5 from above to install and configure metricbeat on each web server in the setup.

On our webserver we’ll enable the following modules

1. Apache
2. System(enabled by default)

1. Enable apache

sudo metricbeat modules enable apache

After the module has been enabled, we can configure the module using /etc/metricbeat/modules.d/apache.yml file.

sudo nano /etc/metricbeat/modules.d/apache.yml

metricbeat.modules:
- module: apache
  metricsets: ["status"]
  period: 10s
  enabled: true

  # Apache hosts
  hosts: ["http://127.0.0.1"]

  # Path to server status. Default server-status
  #server_status_path: "server-status"

  # Username of hosts.  Empty by default
  #username: username

  # Password of hosts. Empty by default
  #password: password

### All configuration should be live reloaded using our live reload configuration

Once everything is configured, log in to your Kibana IP address and go to dashboards and search for metricbeat to look at some awesome dashboards created by default

http(s)://kibana_ip:5601

So, here’s how you can configure metricbeat to send you system metrics data to elasticsearch.

If you want to do some hands-on practice on Elaticsearch, here is the link to my youtube playlist where I show you how to can set up a lab setup with 2 apache servers feeding data to a single-node Elasticsearch cluster deployed on a google cloud platform or GCP.