Collecting metrics in AspNet core app
Hi! Today, I want to explaing to you how you can get metrics from you Web API using Prometheus and Grafana.
- Prometheus is an open-source system monitoring and alerting toolkit. Prometheus collects and stores its metrics as time-series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.
- Grafana allows us to visualize the web application data and metrics collected once connected with the data source.
I’ll try to explain to you how you can dockerize those tools, create a counter for the requests and visualize the data.
Creating the project
Let’s start opening your visual studio and creating a new Web Api project.
Set solution and project name.
Configure .Net version and enable Docker.
Once we have the solution ready, we need to add Prometheus package to the Web Api project. We need to add the following package:
<PackageReference Include="prometheus-net.AspNetCore" Version="5.0.1" />
Now we need to add Prometheus to the pipeline and create a new counter. This is pretty straightforward, we need to go to the “Startup.cs” class and over the “Configure” method add the following code:
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
app.UseSwagger();
app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "WebApi v1"));
}
var counter = Metrics.CreateCounter("webapimetrics", "Counts requests to the WebApiMetrics API endpoints",
new CounterConfiguration
{
LabelNames = new[] { "method", "endpoint" }
});
app.Use((context, next) =>
{
counter.WithLabels(context.Request.Method, context.Request.Path).Inc();
return next();
});
// Use the prometheus middleware
app.UseMetricServer();
app.UseHttpMetrics();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.MapMetrics();
});
}
The code is pretty simple, we are creating a counter named “webapimetrics”, immediately after that we are registering a middleware whose only purpose is to add the method and path to the counter and increase his value.
Docker composing
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.
For more information: https://docs.docker.com/compose/
In order to create a docker-compose file from visual studio, right-click over the web api project and choose “Container Orchestrator Support”.
And choose “Docker Compose”.
And finally select the container OS, in my case I’ll select Linux.
Our project structure will look like this:
Prometheus + Grafana
Prometheus uses prometheus.yml as its configuration file. We need to create this file with the configuration and then add it to the compose.
Let’s override the docker-compose.yml file with the following:
version: '3.4'
networks:
dotnetcoreprometheusgrafana:
services:
webapi:
image: ${DOCKER_REGISTRY-}webapi
build:
context: .
dockerfile: WebApi/Dockerfile
networks:
- dotnetcoreprometheusgrafana
prometheus:
image: quay.io/prometheus/prometheus:latest
depends_on:
- webapi
ports:
- 9090:9090
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
networks:
- dotnetcoreprometheusgrafana
restart: always
grafana:
image : grafana/grafana
depends_on:
- prometheus
ports:
- 3000:3000
restart: always
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin1
- GF_USERS_ALLOW_SIGN_UP=false
networks:
- dotnetcoreprometheusgrafana
I don’t want to explain to you in-depth how this file works but I’ll give you a quick overview of the most relevant things.
- networks: we are creating a new network where the containers will work.
- services: we have 3 services defined, the API, Prometheus and Grafana.
- ports: we are mapping the ports, in this case to the same ones.
- volumes: Volumes are the preferred mechanism for persisting data generated by and used by Docker containers
Under “prometheus” service, we defined a volume. We are saying that we are going to replace “/etc/prometheus/prometheus.yml” with the file in the location “./prometheus/prometheus.yml”, so let’s create it.
Add the following Prometheus configuration:
scrape_configs:
- job_name: webapi-prometheus
scrape_interval: 15s
scrape_timeout: 10s
scheme: http
metrics_path: /metrics
static_configs:
- targets:
- webapi:80
We are defining a new job, setting the scrap interval and the target, where “webapi” is the service and the port defined in the compose file.
Visualizing the data
So, let’s run the app and go to “/metrics”, you will see something like this:
Make several request to the weather endpoint and check the metrics again.
As you can notice, the counter that we created previously is here and is counting the times that we hit the “/WeatherForecast” endpoint.
Now, go to http://localhost:9090 in order to access Prometheus. Here we can perform different queries but I want to show you that we can visualize our counter.
Since Prometheus is receiving the data, we can go to Grafana, http://localhost:3000, and create a new dashboard.
As user we should use “admin” and pass “admin1”. The password was defined in the docker-compose file and you can use whatever you want.
Add a new Prometheus data source.
Configure it with Prometheus URL.
And finally, visualize it.
As always you can check the source code from here.
Enjoy and happy coding! : )