Three Types of Telemetry Data

Distributed Services with Go — by Travis Jeffery (45 / 84)

👈 Chapter 6 Observe Your Systems | TOC | Make Your Service Observable 👉

Observability is a measure of how well we understand our system’s internals — its behavior and state — from its external outputs. We use metrics, structured logs, and traces as the outputs to make our systems observable. While there are three types of telemetry data, each with its own use case we’ll talk about, it’ll often derive from the same events. For example, each time a web service handles a request, it may increment a “requests handled” metric, emit a log for the request, and make a trace.

Metrics

Metrics measure numeric data over time, such as how many requests failed or how long each request took. Metrics like these help to define service-level indicators (SLI), objectives (SLO), and agreements (SLA). You’ll use metrics to report the health of your system, trigger internal alerts, and graph on dashboards to get an idea of how your system’s doing at a glance.

Because metrics are numerical data, you can gradually reduce resolution to reduce the storage requirements and time to query. For example, if we ran a book publishing company, we’d have metrics on each book purchase. To ship a customer’s books, we’d need to know the customer’s order, but after we’ve delivered the books and the return policy has passed, we don’t care about the order anymore. When we’re doing accounting or…