Cloudflare metrics using Prometheus

This story was originally featured on wehkamplabs.com.

The intro

We’ve been using Prometheus for well over a year now, making great use of it’s scraping capabilities within our microservices platform. In fact, the platform doesn’t even accept services without a metrics endpoint for prometheus to scrape. Services generate metrics and also most other parts of our stack get scraped, like Elasticsearch or Kafka. We use Grafana to visualize all these metrics, making them easily accessible to both developers and others who just like to gaze at fancy graphs. Last but not least, the Prometheus alertmanager is used to ping Pagerduty in case of issues somewhere in our stack.

To secure our online presence we chose Cloudflare and we like the graphs and insights they offer us through their web-frontend. But wouldn’t it be nice to have those graphs in our own Grafana instance? Wouldn’t it be nice if we could get alerts based on what happens in Cloudflare? Well, yes. Of course. But that would mean getting info in Prometheus, somehow, right? Yep! And here it is: prometheus cloudflare exporter.

Do the Scraper with me

Things should be pretty straight forward. Create your organization within Cloudflare and add a member with some specific roles:

Strictly speaking, you currently do not need the Raw Logs Access-role since we don’t scrape Cloudflare’s ELS logs yet. We just needed it here for different reasons.

Next you need the Global API Key that comes with this user. Just login to the management dashboard and navigate to my settings where you will find your key.

Now that we’re prepared, it’s time to run the actual exporter. Something like this will probably be enough to make that happen:

docker run \
 -d \
 -p 9199:9199 \
 -e SERVICE_PORT=9199 \
 -e ZONE=example.com \
 -e AUTH_KEY=deadbeefcafe \
 -e AUTH_EMAIL=admin@example.com \
 wehkamp/prometheus-cloudflare-exporter:1.0

There. You now have the prometheus cloudflare exporter online, happily listening on port 9199/metrics and scraping Cloudflare every 60 seconds. The world just got a whole lot better.

We have a visual

The repository at GitHub contains an example Grafana dashboard that can be easily adapted to start querying your own Prometheus instance.

Here’s an example graph showing you cached and uncached requests:

Since we’re scraping the Cloudflare zone analytics per network PoP, we can also present a breakdown of which network pops are used most often when navigating to our website. (Please note these are IATA airport codes)

This clearly shows most of our visitors coming in from the AMS PoP, which would be Amsterdam (NL). Makes sense. With London and Germany at some distance.

And there it is, Cloudflare statistics right on your own doorstep.

The prometheus cloudflare exporter can be found on GitHub and DockerHub.