Monitoring in Production: A Guide to Using OpenTelemetry with NestJS and SigNoz

13 min readMar 12, 2024

In the modern era of software development, monitoring plays a pivotal role in production environments. It ensures the smooth operation of applications, facilitates quick identification of issues, and aids in making informed decisions for improvements. Among the various tools and frameworks available for application monitoring, OpenTelemetry and SigNoz stand out for their powerful tools for application monitoring and observability.

OpenTelemetry is an open-source project that provides a set of APIs, SDKs, and tools for generating, collecting, and exporting telemetry data from applications. This data includes logs, metrics, and traces, which are crucial for understanding the performance and behavior of applications. SigNoz, on the other hand, is an open-source application performance monitoring (APM) tool that leverages OpenTelemetry to collect, store, and visualize telemetry data. Together, they form a powerful duo for achieving full-stack observability in applications.

Understanding OpenTelemetry

OpenTelemetry is a vendor-agnostic and open-source observability framework designed to standardize the collection and export of telemetry data from applications. It supports a wide range of programming languages and cloud providers, making it a versatile choice for developers working with various technologies. The framework is built around several key components, including the OpenTelemetry API, SDK, and Collector, which facilitate the generation, collection, and export of telemetry data to various backends.

One of the significant benefits of using OpenTelemetry is its vendor-neutrality. By standardizing the format and transmission of data, OpenTelemetry enables seamless integration with multiple monitoring and observability platforms. This means organizations are not locked into a specific vendor or proprietary solution, allowing them to choose the best tools for their needs. Additionally, OpenTelemetry’s flexibility in handling data, including the ability to aggregate, enrich, filter, and process telemetry data before exporting it, optimizes data flow and reduces the resource impact on instrumented applications.

Moreover, OpenTelemetry’s simple setup and well-documented APIs streamline the process of instrumenting applications and services for observability. This reduces the complexity and development effort required to gather telemetry data, allowing developers to focus more on their application logic and performance. Despite its numerous benefits, it’s important to note that OpenTelemetry primarily focuses on collecting and exporting traces, metrics, and logs, and does not currently support other data types such as application security events or code profiling information.

Understanding SigNoz

SigNoz emerges as a pivotal tool in the realm of application performance monitoring (APM) and observability, offering a comprehensive solution for visualizing and analyzing telemetry data generated by OpenTelemetry. As an open-source APM tool, SigNoz is designed to be a full-stack observability platform, providing a unified view of logs, traces, and metrics within a single application. This makes it an attractive alternative to proprietary tools like DataDog and NewRelic, offering a self-hosted and open-source solution that addresses privacy and security concerns while providing complete control over configuration and sampling.

SigNoz is built to be OpenTelemetry-native, meaning it is designed to work seamlessly with the telemetry data generated by OpenTelemetry. This compatibility ensures that any application instrumented with OpenTelemetry can easily send its telemetry data to SigNoz for analysis and visualization. The platform supports a wide range of programming languages and frameworks, thanks to OpenTelemetry’s extensive language support. This makes SigNoz a versatile choice for developers working with various technologies, from Python and Java to Node.js and Go.

Features of SigNoz

SigNoz offers a range of features designed to simplify application monitoring and observability:

Metrics Monitoring: Track key metrics such as p99 latency, error rates, external API calls, and individual endpoint performance. This allows for quick assessment of service health and identification of potential issues.
Distributed Tracing: Utilize trace data to pinpoint the exact code causing issues in our application. SigNoz dashboards make it easy to visualize traces with flamegraphs, providing insights into how requests perform across services.
Custom Dashboards: Allows users to build custom dashboards to monitor specific aspects of our application, such as application latency, requests per second, and error percentages.
Alerting: Provides the capability to set up alerts based on log attributes, ensuring timely notification of critical issues.
Service Maps: Quickly assess the health of our services with service maps, providing a visual representation of our application’s architecture and dependencies.
Log Filtering and Querying: Enables users to filter and query logs based on attributes, facilitating efficient troubleshooting and analysis.

SigNoz’s ease of use, coupled with its powerful features, makes it an excellent choice for developers looking to enhance their application monitoring capabilities. Whether you’re running a small-scale web application or a large-scale micro-services architecture, SigNoz offers a comprehensive solution for monitoring and observability needs.

Setting Up Signoz

Setting up SigNoz, a full-stack open-source application monitoring and observability platform, is a straightforward process that can be tailored to both cloud and self-hosted environments. SigNoz offers a seamless way to visualize metrics, traces, and logs, providing a comprehensive view of our application’s performance. This guide will walk you through the installation process and highlight the ease of use and features of SigNoz, including its RED metrics charts and visualization capabilities.

Installing SigNoz

To get started with SigNoz, you can follow these simple steps:

1. Clone the SigNoz Repository: First, clone the SigNoz repository from GitHub to our local machine. This can be done using the following command:

git clone -b main https://github.com/SigNoz/signoz.git

2. Navigate to the Deploy Directory: Change the directory to the deploy folder within the cloned repository:

cd signoz/deploy/

3. Run the Install Script: Execute the install.sh script to start the installation process. This script will automatically install Docker Engine on Linux and set up SigNoz:

./install.sh

On macOS, Docker Engine must be manually installed before running the script.

Once the installation is complete, you can access the SigNoz UI at http://localhost:3301. The dashboard will display a sample application called HOT R.O.D, which comes bundled with the SigNoz installation package, to help you get started.

Instrumenting a NestJS Application with OpenTelemetry

Instrumenting a NestJS application with OpenTelemetry involves a series of steps to ensure that our application can generate and export telemetry data, such as traces, metrics, and logs, which can then be visualized and analyzed using tools like SigNoz. This process enhances our application’s observability, allowing you to monitor its performance and troubleshoot issues more effectively.

Step 1: Install OpenTelemetry Packages

To begin, we need to install the necessary OpenTelemetry packages for NestJS application. These packages provide the tools and functionalities needed to instrument our application, generate telemetry data, and export it to a backend of our choice. Run the following commands in our project directory:

npm install --save @opentelemetry/auto-instrumentations-node
npm install --save @opentelemetry/exporter-trace-otlp-http
npm install --save @opentelemetry/resources
npm install --save @opentelemetry/api
npm install --save @opentelemetry/sdk-node
npm install --save @opentelemetry/semantic-conventions

These packages include:

@opentelemetry/auto-instrumentations-node for auto-instrumenting our Node.js application.
@opentelemetry/exporter-trace-otlp-http for exporting trace data using OTLP (OpenTelemetry Protocol) over HTTP.
@opentelemetry/resources for defining and attaching metadata about our application.
@opentelemetry/api for providing everything needed to interact with the OpenTelemetry API, including all TypeScript interfaces, enums, and no-op implementations.
@opentelemetry/sdk-node for providing tools and functionalities to instrument, generate, collect, and export data.
@opentelemetry/semantic-conventions for standardized naming conventions and constants for telemetry attributes in OpenTelemetry.

Step 2: Configure OpenTelemetry Collector

After installing the necessary packages, you need to configure the OpenTelemetry Collector to receive the trace data and send it to our tracing analytics backend. This involves setting up the endpoint for SigNoz cloud, which is where our application will send the tracing data. If you’re running SigNoz locally, the endpoint might look like http://localhost:4318/v1/traces. For cloud-based SigNoz, you'll need to use the appropriate URL provided by SigNoz.

https://ingest.{region}.signoz.cloud:443/v1/traces is the default url for sending your tracing data to SigNoz cloud. {region} will be your SigNoz data region. You can find these details in SigNoz dashboard under settings --> ingestion settings.

Here’s an example of how to configure the OpenTelemetry Collector to receive the trace data and send it to our tracing analytics backend.

Create a tracer.ts file in source directory and use this sample configuration of OpenTelemetry Collector.

// tracer.ts

"use strict";

import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { Resource } from "@opentelemetry/resources";
import * as opentelemetry from "@opentelemetry/sdk-node";
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";

// Configure the SDK to export telemetry data to the console
// Enable all auto-instrumentations from the meta package
const exporterOptions = {
  url: 'https://ingest.{region}.signoz.cloud:443/v1/traces',
};

const traceExporter = new OTLPTraceExporter(exporterOptions);
const sdk = new opentelemetry.NodeSDK({
  traceExporter,
  instrumentations: [getNodeAutoInstrumentations()],
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: "nestjs-app",
  }),
});

// initialize the SDK and register with the OpenTelemetry API
// this enables the API to record telemetry
sdk.start();

// gracefully shut down the SDK on process exit
process.on("SIGTERM", () => {
  sdk
    .shutdown()
    .then(() => console.log("Tracing terminated"))
    .catch((error) => console.log("Error terminating tracing", error))
    .finally(() => process.exit(0));
});

export default sdk;

Step 3: Start the Tracer

To start the tracer, you need to import and start it in our application’s main file, typically main.ts. This ensures that the tracer is initialized when our application starts, allowing it to begin collecting and exporting telemetry data. Here's an example of how to do this:

// main.ts
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import tracer from './tracer'; // Ensure you have a tracer configuration file

async function bootstrap() {
 tracer.start(); // Start the tracer

 const app = await NestFactory.create(AppModule);
 await app.listen(3000);
}
bootstrap();

This setup ensures that our NestJS application is instrumented with OpenTelemetry, and the telemetry data is sent to SigNoz for analysis and visualization.

Step 4: Verify in SigNoz

Once our application is instrumented and running, you can verify the data in the SigNoz dashboard. You may need to generate some load on our application to see data reported on the SigNoz dashboard. Refresh our application a few times and wait for a few minutes. You should start seeing metrics for latency, rate, and error percentages on the SigNoz dashboard, as well as tracing data in the “Traces” tab.

This is one of my project’s Signoz Dashboard. The application I used is named questify-api

The first look of Signoz monitoring feature.

By following these steps, you’ve successfully instrumented our NestJS application with OpenTelemetry and configured it to send telemetry data to SigNoz. This setup provides a powerful foundation for monitoring your application’s performance and troubleshooting issues in real-time.

Visualizing Data in SigNoz

Visualizing data in SigNoz, especially when it comes to monitoring NestJS applications instrumented with OpenTelemetry, offers a powerful way to analyze and understand the performance of our application. SigNoz provides out-of-the-box RED metrics charts and visualization capabilities, making it easier than ever to identify and resolve latency issues.

Understanding RED Metrics

RED metrics are a crucial aspect of monitoring application performance. They stand for:

Rate of Requests: The number of requests our application is handling per second.
Error Rate of Requests: The percentage of requests that result in errors.
Duration Taken by Requests: The time it takes for a request to be processed and responded to.

These metrics provide a high-level overview of our application’s health and performance, allowing us to quickly identify areas that may need attention.

Viewing Traces at a Particular Timestamp

SigNoz allows us to select a specific timestamp where latency is high and drill down into the traces associated with that timestamp. This feature is invaluable for identifying the root cause of performance issues. By examining the traces at the exact moment a latency spike occurs, we can pinpoint the operations or services that are contributing to the delay. This granular level of detail is essential for effective troubleshooting and optimization

*View of traces at a particular timestamp*

Using Flamegraphs to Identify Latency Issues

When we notice high latency in our application, SigNoz allows us to drill down into specific timestamps to examine the traces associated with those requests. Flamegraphs are a powerful visualization tool that can help we pinpoint the exact cause of the latency. By analyzing the flamegraphs, we can see where the most time is being spent during the processing of a request, allowing us to optimize those areas for better performance.

*Visualize tracing data with Flamegraphs to measure application latency, requests per sec, and error percentage.*

Beyond Visualization: Log Management

In addition to metrics and trace visualization, SigNoz also supports log management. For Node.js applications, such as those built with NestJS, we can use the winston logger to send logs to SigNoz. This integration allows us to correlate log messages with specific requests and traces, providing a more holistic view of our application's behavior. By combining log data with metrics and traces, we can gain a comprehensive understanding of your application's performance and behavior, enabling more informed decision-making and optimization efforts. Here’s how we can set it up:

Install Winston and SigNoz Logger: First, ensure we have winston installed in our project. If not, we can install it using npm:

npm install winston

Additionally, we’ll need to install the SigNoz logger, which is designed to work with winston and send logs to SigNoz:

npm install @signoz/logger

2. Configure Winston with SigNoz Logger: Next, configure winston to use the SigNoz logger. This involves creating a winston configuration file (e.g., logger.ts) and setting up the SigNoz logger as a transport:

import winston from 'winston'
import SignozLogger from '@signoz/logger'

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.Console(),
    new SigNozLogger({
      // SigNoz configuration
      url: 'http://localhost:3301', // Replace with your SigNoz URL
      serviceName: 'nestjs-app',
      // Additional SigNoz options
    }),
  ],
});

export default logger;

3. Use the Logger in Your Application: Now, we can use the configured logger in our NestJS application to log messages. These logs will be sent to SigNoz, where we can view them alongside our metrics and traces:

import logger from '@/logger';

logger.info('This is an informational message');
logger.error('This is an error message');

Once our application is sending logs to SigNoz, we can access the SigNoz dashboard to view these logs. SigNoz provides a searchable interface for logs, allowing you to filter logs by timestamp, log level, and other criteria. This feature is particularly useful for diagnosing issues that may not be immediately apparent from metrics or traces alone.

Monitoring HTTP Endpoints

SigNoz enables the monitoring of HTTP endpoints through the use of the HTTP Check Receiver, a component of the OpenTelemetry Collector. This receiver periodically sends HTTP requests to specified endpoints and collects metrics about the responses, such as response time, status code, and error messages. This setup is particularly useful for synthetic checks against HTTP endpoints, providing immediate visibility into the health of your application’s API endpoints.

To set up HTTP endpoint monitoring with SigNoz, we need to configure the OpenTelemetry Collector with the HTTP Check Receiver and the OTLP exporter to send data to SigNoz Cloud, which we had done before. This involves specifying the target endpoints, the method (GET, POST, etc.), and the collection interval. Additionally, we need to configure the OTLP exporter with the appropriate endpoint for your SigNoz cloud region and include our SigNoz ingestion key for authentication (if we’re using SigNoz Cloud).

Once configured, the HTTP Check Receiver generates metrics for each HTTP response status class, with a value of 1 if the status code matches the class. This allows us to monitor the health of your HTTP endpoints in real-time, identifying any failures or issues that may arise. We can then use the SigNoz dashboard to visualize these metrics, set up alerts for specific failure status codes, and analyze the performance of your endpoints over time.

Exception Tracking

SigNoz also offers automatic exception tracking, providing immediate visibility into errors that occur in our application. This feature is crucial for quickly identifying and troubleshooting issues, as it captures exceptions across various programming languages, including JavaScript for Node.js applications. When an exception is recorded, SigNoz includes rich context, such as stack traces and related request data, to help pinpoint the root cause.

To implement exception monitoring with SigNoz, ensure that our application is instrumented with OpenTelemetry. Then, configure the OpenTelemetry SDK to capture exceptions automatically. This typically involves setting up an error handler that forwards exceptions to SigNoz. Once exceptions are being sent to SigNoz, we can use the SigNoz dashboard to monitor exceptions in real-time, configure alerts based on exception severity or frequency, and analyze exception trends to identify recurring issues. Here’s how we can set it up:

// import relevant opentelemetry functions
import { trace, SpanStatusCode } from "@opentelemetry/api";

...

// Get the current span from the tracer
const span = trace.getActiveSpan();
// Create a new sample error
err = new Error("This is a sample error");
// recordException converts the error into a span event. 
span.recordException(err);
// Update the span status to failed.
span.setStatus({ code: SpanStatusCode.ERROR, message: String(err) });

SigNoz’s exception tracking capabilities are integrated with trace data, allowing us to see exceptions in the context of the full request lifecycle. This integration enhances the ability to understand the impact of exceptions on our application’s performance and user experience, facilitating more effective troubleshooting and resolution.

Conclusion

In conclusion, the combination of OpenTelemetry and SigNoz offers a powerful and flexible approach to monitoring and observability for NestJS applications. Whether you’re running a small-scale web application or a large-scale micro-services architecture, SigNoz provides a comprehensive solution for monitoring and observability needs. By leveraging OpenTelemetry for collecting and exporting telemetry data and SigNoz for visualizing and analyzing this data, developers can gain deep insights into their application’s performance and behavior, facilitating more efficient and effective application management.

OpenTelemetry makes it very convenient to instrument your NestJS application, and you can then use an open-source APM tool like SigNoz to analyze the performance of our app. As SigNoz offers a full-stack observability tool, we don’t have to use multiple tools for our monitoring needs. Whether you choose to run SigNoz cloud for ease of access or opt to install and self-host SigNoz for more control, the choice is yours. With over 16,000+ GitHub stars (as of March 13, 2024), open-source SigNoz is loved by developers, making it a preferred choice for those looking to enhance their application monitoring capabilities.

In the end, the journey of monitoring and observability with OpenTelemetry and SigNoz is not just about collecting data; it’s about making informed decisions that lead to better application performance, user satisfaction, and ultimately, a successful product.