Monitoring Gunicorn with Prometheus

Damian Myerscough
Sep 9, 2018 · 3 min read

Gunicorn is a Python WSGI HTTP server that can be configured to serve your Python based web application. When running Gunicorn; you can enable telemetry to gain a greater insight into how your web application is being served. For example, you can monitor the number of HTTP status codes, number of requests per second, number of Gunicorn workers, request duration, and a few other metrics.

As of Gunicorn version 19.1, statsd telemetry was introduced; so if you want Gunicorn telemetry, you will need to make sure you have a minimum version of 19.1. However, I strongly recommend moving to a version greater than 19.5 for security reasons [1].

In order for Prometheus to be able to process Gunicorn statsd telemetry, you need to leverage the Prometheus statsd-exporter, which formats statsd metrics into Prometheus formatted metrics. First setup and install Gunicorn and write a simple Hello World application in Flask to demonstrate gathering Gunicorn telemetry.

Configure statsd-exporter

We will use Docker to run the statsd-exporter. I assume you already have Docker installed; if so, pull the latest version of the statsd-exporter using the following command.

$ docker pull prom/statsd-exporter

Once the statsd-exporter image has been downloaded, you need to create a metrics map, which allows Prometheus to label the HTTP status codes and enables you to filter out HTTP status codes using a simple regular expression.

After you have created the statsd.conf configuration file, start the statsd-exporter using the following Docker command.

$ docker run -dP --net=host -v ${PWD}/statsd.conf:/statsd/statsd.conf prom/statsd-exporter "-statsd.mapping-config=/statsd/statsd.conf"

Gunicorn and Flask

Here we are creating a simple “Hello World” web application to demonstrate Prometheus capturing Gunicorn telemetry. First, install Gunicorn and Flask; this can be done by executing the following commands.

$ python3 -m venv ~/venv
$ source ~/venv/bin/activate
$ pip install gunicorn flask

After installing Gunicorn and Flask, create a file hello.py that returns a simple “Hello World” string.

You can test hello.py using Gunicorn with the following command and send metrics to our statsd-exporter.

$ gunicorn --statsd-host=localhost:9125 --statsd-prefix=helloworld --bind 127.0.0.1:8080 hello:app

Configure Prometheus

Now that metrics are being emitted to the statsd-exporter, we can configure Prometheus to scrape statsd-exporter and capture our Gunicorn metrics. In your Prometheus configuration file, you need to add the following job and start and/or reload Prometheus.

- job_name: 'helloworld_gunicorn'  static_configs:
- targets: ['localhost:9102']

While Prometheus starts to capture our Gunicorn metrics, we can generate some simple requests to Gunicorn using the following command.

$ for i in `seq 500`; do curl http://127.0.0.1:8080 && sleep $((RANDOM % 5)); done

After Prometheus has scraped the statsd-exporter, you should see the new metrics appear in Prometheus.

Gunicorn Metrics

You may have noticed that the status code for requests coming into Gunicorn are separated out. This would have made querying metrics a little more difficult, however, since we configure a metrics map earlier, you can actually query the HTTP status code using the following promQL queries.

http_response_code{}  # Lists all status codes
http_response_code{status=~"2.."} # List only the 2XX status codes

Configure Grafana

Once Prometheus has captured metrics, you can create a Grafana dashboard to visualize the Gunicorn metrics as shown below.

Gunicorn Metrics Visualized [2]

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade