Introduction to Monitoring Django Applications with Prometheus and Grafana

Published in

Simform Engineering

6 min readSep 13, 2024

Optimize Django application monitoring using Prometheus and Grafana

Imagine a production-level Django application serving thousands of users simultaneously. Without proper monitoring, you won’t have insights into your application’s performance, health, or potential issues. This is where effective monitoring becomes crucial.

Prometheus and Grafana provide a robust monitoring solution for Django applications. Prometheus collects metrics and manages alerts, while Grafana offers powerful visualization capabilities. Together, they help detect and resolve problems before they impact users.

In this guide, we will explore how to configure and use Prometheus and Grafana to collect metrics from your Django app and visualize them in various ways. We’ll cover the setup process and highlight the benefits of using these tools for monitoring.

What is Prometheus?

Prometheus is an open-source monitoring and alerting system that gathers metrics from monitored targets by scraping HTTP endpoints. It’s highly scalable and designed for reliability. In a Django application, you can use Prometheus to collect metrics on various aspects such as request latency, error rates, and database queries. These metrics provide valuable insights into your application’s performance and health, helping you identify and troubleshoot issues efficiently.

What is Grafana?

Grafana is a powerful open-source platform for creating, exploring, and sharing dynamic dashboards and visualizations of time-series data. It seamlessly integrates with various data sources, including Prometheus, enabling users to monitor and analyze metrics collected from Django applications. With Grafana, you can easily create customized dashboards to visualize key performance indicators, track trends, and identify anomalies for monitoring and optimizing your Django applications effectively.

Prerequisites

Django application that you want to monitor
Docker to run Prometheus and Grafana containers, simplifying the setup process

Dependencies

Install django-prometheus library using this command:

pip install django-prometheus

Make sure to update your requirements.txt file after installing the dependencies. You can use the following command for that:

pip freeze > requirements.txt

Configure Prometheus with Django Application

To your settings.py file, add django_prometheus to your INSTALLED_APPS list.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    "django_prometheus",
    "test_app",
]

Add the Prometheus “before” and “after” middleware to your Django MIDDLEWARE list in your Django project’s settings.py file to ensure proper metrics collection.

MIDDLEWARE = [
    'django_prometheus.middleware.PrometheusBeforeMiddleware',
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django_prometheus.middleware.PrometheusAfterMiddleware',
]

Prometheus Before middleware should be at the top, and Prometheus After middleware should be at the bottom of the list.

To your project’s urls.py file, add the URL endpoint for Prometheus metrics:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('prm/', include('django_prometheus.urls')),
    path('test/', include('test_app.urls')),
]

Dockerfile

FROM python:3.10.2-slim-buster

WORKDIR .

# Install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy the application code to the container
COPY . .

# Expose the port on which the Django application will run
EXPOSE 8006

# Command to run the Django application
CMD ["python", "manage.py", "runserver", "0.0.0.0:8006"]

docker-compose.yml

version: '3.8'

services:
  prome_prac:
    container_name: prome_prac
    build:
      context: .
      dockerfile: ./Dockerfile
    restart: 'on-failure'
    ports:
      - 8006:8006

  prometheus:
    image: prom/prometheus
    restart: 'no'
    volumes:
      - prometheus_data:/prometheus
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
    network_mode: host
  
  grafana:
    image: grafana/grafana
    environment:
      GF_INSTALL_PLUGINS: "grafana-clock-panel,grafana-simple-json-datasource"
    restart: 'no'
    volumes:
      - grafana_data:/var/lib/grafana
    ports:
      - 3000:3000
    depends_on:
      - prometheus
    network_mode: host

volumes:
    prometheus_data: {}
    grafana_data: {}

The Dockerfile specifies the environment and dependencies needed to run your Django application within a Docker container. It installs Python, copies your application code, exposes the necessary port, and defines the command to start the Django server.

On the other hand, docker-compose.yaml defines the services and their configurations for your Docker environment. It includes services for your Django application, Prometheus, and Grafana. This file orchestrates the setup, ensuring that each component is deployed and configured correctly

prometheus.yml

global:
  scrape_interval:     5s
  evaluation_interval: 5s

scrape_configs:
  - job_name: 'prom_prac'
    metrics_path: '/prm/metrics'
    static_configs:
      - targets: ['127.0.0.1:8000']

This prometheus.yml file sets up Prometheus to scrape metrics every 5 seconds (scrape_interval). It defines a job named 'prom_prac' that targets a metrics endpoint at '/prm/metrics' on the localhost's port 8000. This configuration ensures that Prometheus collects metrics from the specified endpoint at regular intervals for monitoring.

The Dockerfile, docker-compose.yaml, and prometheus.yml files should be located in the root directory of the Django project.

We have completed setting up our project. Now, we just need to run the Docker Compose file. Use the command below to start all services.

sudo docker compose up

Our Prometheus, Grafana, and Django services are up and running.

Explore Prometheus

You can check the list of scrape targets (the sources from which Prometheus collects metrics) at http://localhost:9090/targets.

Prometheus targeting a /metrics endpoint

Explore metrics visually with Prometheus! Access http://localhost:9090/ to view dynamic graphs:

Prometheus Graph Interface: Query and Visualize Metrics for Monitoring

You can view all the metrics at http://localhost:8006/prm/metrics.

Explore Grafana

To access the Grafana dashboard, visit http://localhost:3000/login. The default username and password for the Grafana dashboard is “admin.”

Now, simply open the Grafana dashboard and add Prometheus as a data source.

Now, you can start creating dashboards in Grafana:

Go to the Home page.
Click Create Dashboard.
Choose a data source.
Choose the metrics you want to visualize and click Run Queries.
Click Apply.

You can create different types of visualization, as shown below:

Additional tips:

Optimize Scrape Intervals: Adjust the scrape interval in Prometheus according to your application’s needs. While a shorter interval provides more granular data, it can also increase the load on your system. Balance it to ensure efficient monitoring without overloading your infrastructure.
Leverage Grafana Alerts: Use Grafana’s alerting feature to receive notifications about critical issues in real time. Set up alerts for key metrics like high error rates or slow response times to ensure you can quickly address potential problems.
Use Custom Metrics: Beyond the default metrics provided by Django, consider implementing custom metrics specific to your application’s domain. This could include tracking specific business metrics or performance indicators relevant to your application’s success.

Conclusion

Monitoring your Django applications with Prometheus and Grafana provides powerful insights into your system's performance and health. By leveraging Prometheus for collecting and querying metrics and Grafana for visualizing these metrics, you can proactively identify and resolve issues, ensuring a reliable and efficient application. Implementing this monitoring stack helps you maintain optimal performance and make data-driven decisions to improve your application’s overall robustness. Start integrating Prometheus and Grafana today to take your Django application’s monitoring to the next level.

Discover more about Grafana and Prometheus through their official documentation for in-depth understanding and enhanced capabilities.

For more updates on the latest tools and technologies, follow the Simform Engineering blog.
Follow us: Twitter | LinkedIn