Monitoring OpenStack instances with Prometheus

Instance metadata in the Horizon UI
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files: []scrape_configs:
# Scrape Prometheus itself
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
# Scrape OpenStack instances
- job_name: 'openstack'
openstack_sd_configs:
- identity_endpoint: https://openstack.example.com:5000/v2.0
username: simon
project_name: prometheus-lab
password: supersecret
role: instance
relabel_configs:
# Keep only active instances
- source_labels: [__meta_openstack_instance_status]
action: keep
regex: ACTIVE
# Keep only instances which are flagged for scraping
- source_labels: [__meta_openstack_tag_prometheus_io_scrape]
action: keep
regex: 'true'
# Update the scraping port if required
- source_labels:
- __address__
- __meta_openstack_tag_prometheus_io_port
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
# Replace the default instance by the OpenStack instance name
- source_labels: [__meta_openstack_instance_name]
target_label: instance
Node exporter metrics
OpenStack targets in Prometheus
Idle CPU metric
  • If you need to scrape multiple exporters for the same Nova instance (MySQL for instance), you can add another OpenStack SD job to the Prometheus configuration and use different OpenStack metadata keys (prometheus_io_scrape_mysql=true and prometheus_io_mysql_port=9104 for instance) in the relabel_configs rules.
  • In the demo, Prometheus is running within the same subnet as the monitored OpenStack instances. If you want to run Prometheus from the “outside”, the setup gets more complicated but PushProx can probably help.
  • The OpenStack service discovery mechanism is also able to scrape Nova hypervisors. Personally I find it less useful since you usually have some configuration management tool which can inject those hypervisors as static targets in the Prometheus configuration file.
relabel_configs:
# Keep only instances which are flagged for scraping
- source_labels: [__meta_openstack_tag_prometheus_io_scrape]
action: keep
regex: 'true'
...# Keep the target if the prometheus_io_address_pool tag is "mgmt" and it matches with the current subnet.
- source_labels:
- __meta_openstack_address_pool
-__meta_openstack_tag_prometheus_io_address_pool
action: replace
regex: "mgmt;mgmt"
target_label: __tmp_keep_target
replacement: "true"
# Keep target if the prometheus_io_address_pool tag is "internal" and it matches with the current subnet.
- source_labels:
- __meta_openstack_address_pool
-__meta_openstack_tag_prometheus_io_address_pool
action: replace
regex: "internal;internal"
target_label: __tmp_keep_target
replacement: "true"
# If you have more subnets, add more relabel configuration stanzas here.# If the prometheus_io_address_pool tag is empty, use "mgmt" by default.
- source_labels:
- __meta_openstack_address_pool
- __meta_openstack_tag_prometheus_io_address_pool
- __tmp_keep_target
action: replace
regex: "mgmt;;"
target_label: __tmp_keep_target
replacement: "true"
# keep only targets that matched any of the above.
- source_labels: [__tmp_keep_target]
regex: true
action: keep

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

A day in the life of a ReCharge engineer

How to grant multiple users permissions to your EKS cluster

SIEM 101  —  Basic usage

AVATA.Network Whitelist Guide

Build your own custom funnels using BigQuery and Data Studio with Google Analytics

BLE Final

Minimum CSS Every Front End Developer Should Know

ARK Explorer V3 — Developed From The Ground-Up, New Design, Quicker Response and Ready for Push…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Simon Pasquier

Simon Pasquier

More from Medium

Automation of domain expiration checking with prometheus-stack in Kubernetes

Blue-green deployments with Materialize

Kubernetes workflow in short

How to install prometheus and configure it with grafana in ubuntu