Effortlessly simplifying business metrics monitoring
To get better at anything, you need to understand how you’re currently doing. That’s why, at Qonto, as in many companies, we set clear business metrics: measurable values that track and determine the success (or the failure) of our various business operations.
And, to assess these metrics, we have a whole host of tools that we monitor on an ongoing basis, giving detailed insights into our performance.
To give our customers the best experience of our product (one of our five key values is being customer-focused), we need to monitor how our features perform in terms of business impact. This means asking questions like: did the last change improve client acquisition? Are we generating cards fast enough?
Real-time monitoring of business metrics allows us not only to understand the effectiveness of our features, but also to detect anomalies or challenges that could impact our customers — and to react quickly. It also lets us back up our decisions with data and a shared language that all teams understand.
So, for all the reasons above, we built and introduced a new component: Business Metrics Exporter (BME). It’s a tool that’s in charge of collecting information from our internal databases and exposing it in Prometheus metrics.
As Qonto has grown, and the effectiveness of this approach has been confirmed time and again, we have found more situations in which we can use the BME. After being introduced three years ago, our first implementation started having scalability and maintainability issues.
It was time for a revamp!
Back to basics: so what exactly is an exporter?
At the simplest level, an exporter ingests data from a range of sources and produces metrics for monitoring.
Our monitoring stack includes Prometheus as the backend to store and query metrics; it’s a cloud-native monitoring solution that collects application metrics in real time.
By keeping a close eye on these metrics, we can identify trends, anticipate issues before they escalate, and uncover in-the-moment opportunities to improve our services.
For the components that don’t natively expose their metrics in Prometheus-readable format, we need to develop what we call Prometheus Exporters. These bridge the gap between Prometheus and various data sources. A Prometheus Exporter extracts key metrics from databases, services, or hardware, converting them into a format understandable by Prometheus.
Introducing Qonto’s Business Metrics Exporter (BME)
Qonto’s BME pulls data from different databases, converts it into a Prometheus format and exposes it in its /metrics endpoint, which will be consumed by Prometheus.
Here’s a top-line illustration of how the tool works:
N.b. this exporter is not the only one we use at Qonto. Take, for example, our open-source Prometheus RDS exporter which allows Amazon Relational Database Service monitoring.
The problem we encountered? Scalability and maintainability
As with any new tool, we ran into a few challenges when implementing the BME. Since it was developed to cover very specific use cases, we followed an approach where the lifecycle of each metric was coded independently in the programming language Go. Once we saw the effectiveness of tracking these metrics in Prometheus, we started adding more and more use cases. The result? We needed a more scalable strategy.
This first code-driven approach was a big undertaking, involving digging into the codebase as well as facing a lot of duplicated code, as for each exposed metric, we needed:
- to manage database connection,
- a collector to implement the Prometheus Collector interface,
- a repository to fetch data from the database and store the information,
- to instantiate those collectors and repositories in the main function.
All of those steps represented around 600 lines of duplicated code per metric. Even though a collector could gather many metrics, which reduced the steps, the integration of a new metric involved a lot of time and effort.
Finding the solution: moving to a configuration-driven approach
Our journey towards a more scalable and maintainable solution led us to develop a configuration-driven exporter. This meant allowing our teams to define, manage, and modify metrics through simple YAML configuration files. And the benefit? No more code duplication or wasted time.
Additionally, this approach makes it easier for everyone to get involved in our business metrics collection, especially since often the people that are the most interested in these metrics aren’t Go developers. Since it’s a YAML file, if you know the query that gets your data, you don’t need to touch any Go!
Moving away from our original code-based approach, this is the configuration code that needs to be implemented within the service to add a new metric:
job:
- name: 'transfers-status-collection' # job name
schedule: '*/10 * * * *' # every 10 minutes
source: 'postgres://{{DB_USERNAME}}:{{DB_PASSWORD}}@{{DB_HOST}}/{{DB_NAME}}'
metrics:
- name: 'transfers_status' # metric name
subsystem: 'core'
help: 'Provides the number of transfer by status'
labels:
- status
value: count
query: |
SELECT COUNT(*) AS count, status
FROM transfers
WHERE status IN ('to_send', 'failed')
GROUP BY(status);Why this matters: the impact of our evolution
Scalability
Moving to a configuration-based system has significantly improved our ability to scale. With the autonomy to add and adjust metrics as needed, teams can ensure our monitoring capabilities evolve, along with our business requirements.
Maintainability
Ease of maintenance is the second benefit of this evolution. Imagine only needing to update a YAML file to adjust or add metrics. No more deep dives into complex code. But there’s more: we’ve built a one-time setup for connecting to databases, fetching data, formatting it for Prometheus, and smart caching. This setup is tested and ready to go.
Increased adoption
Since your presentation of the new configuration approach during the Backend Monthly, our team wants to use this exporter.
— Gustavo, backend engineer
Simplifying the process of adding and modifying metrics has resulted in a significant increase in adoption across teams. More teams have used this exporter in the last few months than in the first three years of its existence. Before the refactoring, only 15% of our Backend teams had adopted the exporter, while the percentage more than doubled to 31% in one month with the new approach. And that figure keeps growing!
Key learnings
- Monitoring business metrics is crucial for understanding operational efficiency, customer satisfaction, and financial health.
- Exporters are essential tools that translate complex data into actionable insights for monitoring and alerting.
- A configuration-driven approach enhances scalability and maintainability in monitoring business metrics.
- It’s important to provide an easy solution to a complex problem, facilitating teams’ adoption.
What’s next?
- Moving to open source. From our experience, this tool can be very useful for a company and extend to a lot of use cases, so we plan to open source it in the future, meaning that others will be able to benefit from it.
- Service Level Objectives (SLOs). We’ll keep adding new business metrics to our Prometheus setup and leverage on SLO tracking and alerting. This will help us deliver value to our customers, ensuring our operations align with both their expectations and our business goals.
The BME enables a deeper understanding of our customers’ daily interactions with our tools, providing a comprehensive overview and insightful analysis of the technologies we’ve developed. It’s guiding our performance today, but also setting our destination for the future.
About Qonto
Qonto makes it easy for SMEs and freelancers to manage day-to-day banking, thanks to an online business account that’s stacked with invoicing, bookkeeping and spend management tools.
Created in 2016 by Alexandre Prot and Steve Anavi, Qonto now operates in 4 European markets (France, Germany, Italy, and Spain) serving 450,000 customers, and employs more than 1,400 people.
Since its creation, Qonto has raised €622 million from well-established investors. Qonto is one of France’s most highly valued scale-ups and has been listed in the Next40 index, bringing together future global tech leaders, since 2021.
Interested in joining a challenging and game-changing company? Take a look at our job offers.
Illustration by Karina Pasechka
