.Net Core Microservice Metrics with Prometheus and Grafana

Today, we will see how to capture metrics such as CPU Usage, HTTP Request Count etc. for a .Net Core Microservice in Prometheus and display the details in a Dashboard in Grafana.

Before we go to hands-on part,

Why metrics are required?

Metrics, are basically numerical representations of data related to system performance and behavior. These are often used to count or measure a value and are aggregated over a period of time.

Metrics give us insights into the historical and current state of a system. Since they are just numbers, we can also use them to perform statistical analysis and predictions about the system’s future behavior.

You can also use metrics to trigger alerts and notify you about issues in the system’s behavior.

What is Prometheus?

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company. To emphasize this, and to clarify the project’s governance structure, Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project, after Kubernetes.

Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.

Prometheus Architecture (Credit: https://prometheus.io/docs/introduction/overview/)

What is Grafana?

Grafana is a complete observability stack that allows you to monitor and analyze metrics, logs and traces. It allows you to query, visualize, alert on and understand your data no matter where it is stored. Create, explore, and share beautiful dashboards with your team and foster a data driven culture.

So, Today we will create a containerized .Net Core Microservice, that will expose a metric endpoint and we will setup a Prometheus instance, which will capture those metrics and stored in the time series database, then we will visualize the data in Grafana dashboards.

Prerequisites:

1. VS 2022 Community Edition ( you can use other versions as well)
2. Docker Desktop

We will use docker compose to orchestrate the containerization and deployment

Let’s start,

1. Open VS 2022 and Create a new .Net Core Web API project, it will create Weather Forecast API by default. Choose Authentication Type as none.

Our Microservice

2. We need to install a few NuGet packages, please run the below commands in the package manager console,

1. Install-Package prometheus-net
2. Install-Package prometheus-net.AspNetCore
3. Install-Package prometheus-net.AspNetCore.HealthChecks

We need this packages to create a new metrics or to expose default metrics available with the package.

3. Once package are installed, we need to integrate Prometheus Exporter to ASP. Net Core Middleware. We will modify the Startup.cs to configure our middleware to export metrics.

4. Now our Application is ready, if you run the application and go to “/metrics” endpoint, you will be able see the metrics.

Metrics

Now we are able to expose, the application metrics via an endpoint.

5. Let’s start containerizing our Application. Right click on the project in the Visual Studio → Add → Docker Support → Choose Target OS (Linux or Windows, I am doing it for Linux Containers). It will add a Dockerfile in the root directory of the project. This Dockerfile contains, instruction to create the container image for our application.

6. Now, our Image creation is done, we need to deploy our container image, to do that, we will use docker compose. Now again, right click on the project in Visual Studio → Add → Container Orchestration → Choose Docker Compose → Choose Target OS (Linux or Windows, I am doing it for Linux Containers). It will add a new Docker Compose Project in the root directory of the Solution. Once Docker Compose project is added, Please right click on the compose project and add new folders in below structures. Our Project will look like below,

7. In the Docker Compose project, Go to grafana → provisioning → dashboards, Right Click → Add → New Item and add two new items.

Add dashboard.yml and GrafPormAPIDashboard.json

These will create a dashboard while creating the Grafana Container.

8. Now in the Same Compose Project, Go to grafana → provisioning → datasources, right click → Add → New Item and add one new item.

Add prometheus_ds.yml

It will set up a data source in Grafana, while creating the Grafana Container.

Make Sure to use the Container Name, these container name will be available in docker-compose.yml file.

9. Now in the Same Compose Project, Go to prometheus ,right click → Add → New Item and add one new item.

It will create a job at the Prometheus side during container creation, which will monitor and scrape the metrics endpoint of the application and store the data in the Prometheus database. Please use the host name used by docker and host port via which metrics endpoint is exposed, in the targets section of the yaml. For me it is “host.docker.internal”, you can check out that in the host settings file of the OS.( for windows, “C:\Windows\System32\drivers\etc\hosts”)

10. Now Let’s modify the docker compose, to create all the containers. Please checkout the below compose file. This docker-compose.yml, file will create 3 containers, grafpormapi contianing our microservice, grafana_grafporm containing Grafana image and prometheus_grafporm containing Prometheus image.

11. Now we are ready, let’s run the application, by running the docker-compose configuration in Visual Studio.

12. Once the Application is running, it will look like below,

Go to Docker Desktop,

Docker Compose Containers

So, all our 3 containers are up and running.

13. Let’s check our Restful API Microservice,

Swagger UI of the API

Test the API using the Swagger Endpoint. It will also generate the metrics, which we need to test our integration.

14. Now, let’s hit the metrics endpoint and see, (/metrics)

/metrics

Metrics have been exposed successfully.

15. Now let’s go to Prometheus UI and see if the metrics have been scraped successfully. (http://localhost:9090/)

Metrics have been scraped successfully in Prometheus UI

Graph View of metrics in Prometheus UI

16. Now Let’s go to Grafana. (http://localhost:3000/). For initial login use “admin” as username and password. Then Go To → Dashboards →Manage and Select Our Newly Created dashboard “GrafProm.API Metrics Dashboard”

Select the Dashboard

Visualize the Metrics

Now we are done, we are successfully able to export metrics to Prometheus and Visualize in the Grafana Dashboard.

Reference,

1. GitHub Repo: https://github.com/arkapravasinha/GrafProm
2. Prometheus .Net Client Library: https://github.com/prometheus-net/prometheus-net
3. Prometheus: https://prometheus.io/
4. Grafana: https://grafana.com/