UPDATE: master branch is including now monitoring. Versions are bumped and adjusted docker-compose file. Some of the screens/code snippets appearing here may be different now, but won’t stop you to configure everything following the same approach. If you are still finding issues, please let me know ;).
In my previous article I've already written about how to deploy a Ktor server using Docker, and this one will take it as a starting point, so you may be interested in checking it before moving forward:
The next step is to monitor how our server instances are performing.
Tools we need
Let me explain the tools we are going to use and why do we need them:
- Prometheus: it will be responsible of scraping/querying our backend instance to recover metrics. It will also work as Grafana's datasource.
- Micrometer: library that should be included in our backend instance to generate the metrics we need.
- Grafana: monitoring tool to query datasources, visualise data and alert in case of reaching defined thresholds.
Setting containers up!
First of all, let's recall how docker-compose.yml file is defined so far:
Two containers with backend instance and database. Not too complex. Now we need to add two new Docker containers, Grafana and Prometheus:
As you can see, a couple of new containers with their volumes. Just left to mention I've added a prometheus.yml file inside our Prometheus volume host folder. This file will define where Prometheus will query/scrape data from and some other parameters. This is how it looks like:
As you could guess if you read the config file, we need to include a new route to our backend where Prometheus will find metrics (metrics_path).
Metrics and route definition
To prepare metrics in our backend instance we need to add some dependencies to our project. As described before, Micrometer will be the responsible of generating those values and we will use a PrometheusRegistry to transform metrics into Prometheus "language". This is what we need to add to our build.gradle file:
Now that dependencies are in place, it is time to configure that extra route to feed Prometheus with metrics:
As always, this metric's routing should be added to our application when launching our Ktor server:
As you can see at the top of the previous snippet, it is also needed to install a metric's feature. This new feature will need a registry and a list of metric's generators.
It is just left to explain where that PrometheusRegistry comes from. As it is being used in different places, it is defined as a Koin dependency:
Show me some graphs!
At this point, the hard work is done, but there are still a few things to do. Rebuilding backend Docker image and launching Docker compose up is the first in our list. Once these tasks are finished, you should be able to access Grafana control panel using http://localhost:3000 url (if you are running Docker compose in a server, not in your personal computer, I guess it is not needed to explain to you how to access Grafana there ;) )
Grafana will ask you first to input username/password so you can enter default values admin/admin to login. Once logged in, you need to define a datasource (pointing to Prometheus instance) and a dashboard (set of graphs to visualize data).
To define Prometheus datasource, just replicate the config shown in the previous image.
NOTE: please make sure you add http://prometheus:9090 and not the suggested http://localhost:9090.
To define Grafana dashboard, look for import option on the left menu, select "Upload .json file" and upload this file. Grafana will ask you to set a dashboard name and associate a datasource (the one defined in the previous step). Once you save, graphs should be visible in your screen!
Don't forget to send some calls to your backend instance to see the graphs reacting. You can use Postman collection added in the repository.
If you want to give it a try, clone the following Github repository (feature/monitoring branch) and don’t be shy to leave a comment or a PR if you feel you can add something. Both are welcome!
Have fun and happy “clean” coding!
