Comprehensive MongoDB Monitoring with Prometheus and Grafana

Monitoring is a critical component of database administration, and MongoDB is no exception. With the right tools, you can gain valuable insights into your database’s performance and health. Prometheus and Grafana are two open-source projects that, when combined, offer a powerful solution for monitoring MongoDB. Here’s how to set everything up from scratch.

Photo by Resource Database on Unsplash

MongoDB Metrics Observable by Prometheus

To monitor MongoDB with Prometheus, you need to expose MongoDB metrics in a format that Prometheus can scrape. This can be done through the MongoDB Exporter, which translates MongoDB metrics into Prometheus metrics.

Here’s a non-exhaustive list of observable MongoDB metrics that you can monitor with Prometheus:

1. mongodb_up: Indicates whether the MongoDB instance is up and running.
2. mongodb_connections: Number of current connections to the database.
3. mongodb_memory_usage_bytes: Memory usage details, including resident, virtual, and mapped memory.
4. mongodb_network_bytes_total: The total number of bytes sent and received over the network.
5. mongodb_op_counters_total: Database operation counters, split by type (insert, query, update, delete, etc.).
6. mongodb_document_operations_total: Document operations, including documents inserted, updated, deleted, and returned.
7. mongodb_replication_headroom_seconds: The replication headroom in seconds.
8. mongodb_replica_set_member_state: Numeric state of the replica set member.
9. mongodb_storage_engine: Indicates the storage engine in use.
10. mongodb_asserts_total: Counts of various asserts triggered.

These metrics can help in monitoring the health and performance of your MongoDB instance. Now, let’s look at how you can visualize these metrics with Grafana.

Implementing Your Monitoring Solution

To implement this monitoring setup, you’ll need to:

• Configure MongoDB to expose metrics (usually via the MongoDB Exporter).
• Set up Prometheus to scrape these metrics at a regular interval.
• Import your chosen Grafana dashboard template or create a custom one to visualize the data.

Step 1: Set Up the MongoDB Exporter

The MongoDB Exporter is a bridge between MongoDB and Prometheus, converting MongoDB metrics into a format that Prometheus can understand. To set it up:

1. Download MongoDB Exporter: Obtain the latest version from the official repository.
2. Run MongoDB Exporter: Launch the exporter so it can connect to your MongoDB instance. For a local instance, use:

/mongodb_exporter --mongodb.uri=mongodb://localhost:27017

This command starts the exporter and exposes the metrics on port 9216.

1. Secure the Exporter: Make sure the exporter is accessible only to Prometheus, either by using firewall rules or by running it on a private network.

Step 2: Configure Prometheus to Scrape MongoDB Metrics

With the exporter up and running, the next step is to configure Prometheus to scrape the exposed metrics:

1. Modify prometheus.yml: Add a new job under scrape_configs:

scrape_configs:
  - job_name: 'mongodb'
    static_configs:
      - targets: ['<EXPORTER_HOST>:9216']

1. Replace <EXPORTER_HOST> with your MongoDB Exporter's hostname or IP.
2. Reload Prometheus Configuration: Apply the new configuration by either restarting Prometheus or hitting the /-/reload endpoint.

Step 3: Visualize Metrics with Grafana

With Prometheus collecting the metrics, use Grafana to create visualizations:

1. Log in to Grafana: Access the Grafana web interface, usually at http://<GRAFANA_HOST>:3000.
2. Add Prometheus as Data Source: Go to “Configuration” → “Data Sources” → “Add data source”. Choose “Prometheus” as the type, set the Prometheus server URL, and save the configuration.
3. Import Dashboards: Select “Import” from the “+” menu, and enter one of the dashboard IDs mentioned earlier, such as 2583 for a general MongoDB overview. After the import, select the Prometheus data source you added.

Select Grafana Dashboard Templates for MongoDB

Once your data source is connected, consider using these popular templates that can help you get started with MongoDB monitoring:

1. MongoDB Overview Dashboard (ID: 2583)
   Offers a comprehensive overview of MongoDB metrics, including operations, connections, memory, and network usage.
2. MongoDB ReplSet Dashboard (ID: 7373)
   Focused on MongoDB replica sets, providing insights into replication lag, the state of replicas, and more.
3. MongoDB Atlas Cluster Overview (ID: 8785)
   Tailored for MongoDB Atlas users, this dashboard includes metrics relevant to clusters hosted on MongoDB’s managed service.
4. MongoDB Prometheus Exporter Overview (ID: 11159)
   This dashboard is designed specifically for visualizing metrics exposed by the MongoDB Exporter for Prometheus.
5. MongoDB Performance Dashboard (ID: 10012)
   A deep-dive into performance metrics, suitable for troubleshooting and performance analysis.

These templates provide a variety of views into your MongoDB metrics, from general overviews to deep dives into performance statistics.

Conclusion

By integrating MongoDB with Prometheus and Grafana, you’ve created a robust monitoring solution that can provide a comprehensive view of your database’s health and performance. Remember to tailor your dashboards to your specific needs and keep your Prometheus and Grafana instances up to date.

About the Author

Adam Anderson is a passionate software engineer with more than 10 years of experience in C/C++, Java, and Python application development. He has a strong interest in build automation, DevOps practices, and project management. When not diving into code and configuration files, Adam enjoys hiking in the great outdoors and exploring new technology trends. You can reach out to Adam Anderson via email at xsizxenjin@gmail.com for more insights on software development and project management.

Your support is appreciated — will at a minimum empower me to dig in more to motivate education, learning initiatives as well as to generate and promote productization ideas

• Tips $1 https://buy.stripe.com/14keXDeWx0ML1j27ss
• Subscription $5 https://buy.stripe.com/3cs2aR6q1cvt2n6fYZ
  https://medium.com/@xsizxenjin/subscribe/

Thank you!!

Photo by Alexas_Fotos on Unsplash

Miscellaneous Selected Articles:

1. Unleashing the Power of `awk`: Advanced Text Processing Techniques
2. Mastering Data Processing with jq: Advanced Techniques for MongoDB
3. Maven Packaging: ZIP vs. WAR — A Comprehensive Comparison
4. Enhancing Java Application Security: A Deep Dive into the Maven Dependency Plugin’s Top 20 Vulnerability Finds
5. 20 Advanced Pipelining Tips and Tricks Using AWS CLI