Getting started with Metricbeat
How to monitor system metrics data using Metricbeat and elasticsearch.
IT systems can be monitored for various attributes such as performance, reliability, availability etc. Metrics is the measurement of behaviour and usage of system resources that can be collected and monitored from the system.
In today’s day and age of highly complex IT systems and environments, monitoring of system metrics helps increase the availability and reliability of the system and allows IT teams to quickly respond to any fault if they occur.
ELK stack became elastic stack with the addition of beats, a family of light-weight shippers used for different use cases and data sets. In the previous articles, I gave an overview of the elastic stack and installed an elastic search on Linux. In this article, I’ll focus on Metricbeat.
Metricbeat is a lightweight agent that can be installed on target servers to periodically collect metric data from your target servers, this could be operating system metrics such as CPU or memory or data related to services running on the server. It can also be used to monitor other beats and ELK stack itself. It is built on top of libbeat — a GO framework.
It can be configured to send output to elasticsearch directly or to logstash if you need to transform the data.
Lab Setup
In this article, I’ll set up a single-node elastic search cluster(refer to this article) and two apache webservers. I have used the GCP platform to build my test lab since it offers $300 USD free trial credit but you can do it on your servers or any other public cloud platform as well.
We’ll use metricbeat’s internal collection method to collect monitoring data about our elasticsearch cluster.
We’ll also setup metricbeat to collect metric data from our webservers and send it to elasticsearch.
Metricbeat Installation
Note: You’ll need an existing elasticsearch cluster to store metrics data and Kibana to visualise this data. Please read this article to help set up a single-node elasticsearch instance and Kibana.
We’ll use the APT repository method to install Metricbeat.
Installation and configuration of Metricbeat on ELK Server
- Enable Monitoring on Elasticsearch Cluster
Before we can install metricbeat to monitor our ELK cluster, we first need to enable monitoring in our elasticsearch.yml file
cd /etc/elasticsearchsudo nano /etc/elasticsearch/elasticsearch.ymlxpack.monitoring.collection.enabled: truexpack.monitoring.elasticsearch.collection.enabled: true
2. Restart Elasclearticsearch service
sudo systemctl restart elasticsearch
3. Install Metricbeat
Note: Steps 3.1–3.3 are not necessary if you have previously set up your APT repository during elasticsearch deployment
3.1. Download and install public signing key
wget -qO — https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
3.2. Install apt-transport-https package
sudo apt-get install apt-transport-https -y
3.3. Save directory definitions
echo “deb https://artifacts.elastic.co/packages/7.x/apt stable main” | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
3.4. Install Metricbeat
sudo apt-get update && sudo apt-get install metricbeat
4. Configure Metribeat
Metricbeat configuration can be found in /etc/metricbeat/metricbeat.yml file. Please be careful while configuring this file since YAML is syntax sensitive. Please feel free to read this article to learn a bit more about YAML.
sudo sunano /etc/metricbeatmetricbeat.yml### live monitoring sectionreload.enabled: truereload.period: 10s### name and tagname: elk-metricsTags: demo-elkenv: lab### enable kibana dashboardsSetup.dashboards.enabled: truehost: <<ip_address_of_elastic_server>>:5601### output configurationoutput.elasticsearch:# Array of hosts to connect to.hosts: [“<<ip_address_of_elastic_server>>:9200”]# Protocol — either `http` (default) or `https`.protocol: “https”username: “elastic” #### elasticsuperuserpassword: “password”. #### elastic users password## logginglogging.level: debuglogging.selectors: [“*”]logging.to_files: truelogging.files:path: /var/log/metricbeatname: metricbeatkeepfiles: 7permissions: 0644## enable interal collection of metricbeatmonitoring.enabled: truemonitoring.elasticsearch
one configuration I’d like to point out in the metricbeat.yml configuration file is live reload
reload.enabled: true
reload.period: 10s
this config allows the user to make changes to module configurations files without the need to restart metricbeat service. Metricbeat checks if there have been any changes made to the module configuration filed based on the timer set in reload.period setting.
5. Start Metricbeat
sudo systemctl start metricbeat
6. Modules and metric sets
Metricbeat does its magic by using modules and metric sets. Let's try and understand these:
Let's say you want to monitor one of your Linux servers for its CPU, load, memory etc. what you’ll need to do is after you install metric beat on the VM you need to enable a module called system which provides system monitoring information. The system module allows you to monitor a lot of things as listed below
metricsets:
— cpu # CPU usage
— load # CPU load averages
— memory # Memory usage
— network # Network IO
— process # Per process metrics
— process_summary # Process summary
— uptime # System Uptime
— socket_summary # Socket summary
#- core # Per CPU core usage
#- diskio # Disk IO
#- filesystem # File system usage for each mountpoint
#- fsstat # File system summary metrics
#- raid # Raid
#- socket # Sockets and connection info (linux only)
#- service # systemd service information
Each of these items individually is called a metric set. You can either choose to enable all of these or only the ones that you are interested in.
6.1 Configure Modules for ELK servers
For our elasticsearch cluster, we’ll configure
- elasticsearch-xpack
- Kibana-xpack
6.1 Enable elasticsearch-xpack
First, we’ll need to enable the module using the following
sudo metricbeat modules enable elasticsearch-xpack
After the module has been enabled, it can be configured using /etc/metricbeat/modules.d/elasticsearch-xpack.yml file
nano /etc/metricbeat/modules.d/elasticsearch-xpack.yml- module: elasticsearch
xpack.enabled: true
period: 10s
hosts: ["http://localhost:9200"]
#scope: node
username: "remote_monitoring_user"
password: "remote_monitoring_user_password"
#ssl.enabled: true
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
#ssl.certificate: "/etc/pki/client/cert.pem"
#ssl.key: "/etc/pki/client/cert.key"
#ssl.verification_mode: "full"
xpack.enabled: true
6.2 Enable Kibana-xpack
As we’ve done earlier, first we’ll need to enable Kibana-xpack module
sudo metricbeat modules enable kibana-xpack
After the module has been enabled, it can be configured using /etc/metricbeat/modules.d/kibana-xpack.yml file
nano /etc/metricbeat/modules.d/kibana-xpack.yml- module: kibana
metricsets:
- stats
period: 10s
hosts: ["localhost:5601"]
#basepath: ""
#username: "remote_monitoring_user"
#password: "remote_monitoring_user_password"
xpack.enabled: true
Installation and configuration of Metricbeat on WebServer
Note: Repeat steps 1–5 from above to install and configure metricbeat on each web server in the setup.
On our webserver we’ll enable the following modules
- Apache
- System(enabled by default)
1. Enable apache
sudo metricbeat modules enable apache
After the module has been enabled, we can configure the module using /etc/metricbeat/modules.d/apache.yml file.
sudo nano /etc/metricbeat/modules.d/apache.yml
metricbeat.modules:
- module: apache
metricsets: ["status"]
period: 10s
enabled: true
# Apache hosts
hosts: ["http://127.0.0.1"]
# Path to server status. Default server-status
#server_status_path: "server-status"
# Username of hosts. Empty by default
#username: username
# Password of hosts. Empty by default
#password: password
### All configuration should be live reloaded using our live reload configuration
Once everything is configured, log in to your Kibana IP address and go to dashboards and search for metricbeat to look at some awesome dashboards created by default
http(s)://kibana_ip:5601
So, here’s how you can configure metricbeat to send you system metrics data to elasticsearch.
If you want to do some hands-on practice on Elaticsearch, here is the link to my youtube playlist where I show you how to can set up a lab setup with 2 apache servers feeding data to a single-node Elasticsearch cluster deployed on a google cloud platform or GCP.