Storing All of Your Observability Signals in One Place Matters!
Going from “Swivel Chair” Observability to “Single Pane of Glass” Observability
In Observability’s early days, there was often a lot of talk about the “three pillars”. That is, traces, logs, and metrics, which gave us the information to make our systems observable. The problem with referring to these three signals as “pillars”, is that it implies that they’re siloed and therefore independent of each other, when in fact, the exact opposite is true.
To harness the true power of Observability, you need to treat these signals not as pillars, but as three strands that make up a braid, as OpenTelemetry (OTel) co-founder (and my former teammate) Ted Young so aptly put it. This means that traces, logs, and metrics, each of which serves a different and important purpose, contribute to the observability story, giving us the full picture of what’s happening in our systems. The signals must also be correlated. Because what use is a metric or a log if you don’t know what trace it pertains to?
And yet, many organizations practicing Observability today still tend to send different signals to different backends for storage and analysis. The problem with this setup is two-fold. First, you’re not storing all of the signals in one place. Second, you’re having to go back and forth between different tools to look at your signals, try to correlate them, and understand what’s going on. How can you effectively analyze all of your data if it’s not all stored in the same place?
This problem is further amplified when you consider how some organizations will send telemetry data from different applications to different vendors. For example, an organization might have App A send metrics to SaaS Tool X, and traces and logs to SaaS Tool Y. And App B sends traces SaaS Tool J, logs to SaaS Tool L, and metrics to self-hosted Tool M. Let’s not forget the teams that have gone rogue and decided to do their own thing. See that Dell tower under Bob’s desk? Wouldn’t you know it? It’s running a bunch of OSS self-hosted tools, and App C is sending telemetry data there. Wheeee!
Oh boy…audit will have a field day with that one. To borrow a term coined by my husband, these types of organizations are practicing “swivel chair” observability, and to be honest, calling it “observability” at this point is being generous. You don’t have a braid. You don’t even have pillars. You’ve got islands of pillars. Zoinks!
Instead, what you want (and need) is a “single pane of glass” for your Observability. One place where your signals are stored. One place where you can view your signals. One place where you can understand how your signals are correlated.
In doing so, you can fulfill the promise of Observability, enabling you to:
“Ask meaningful questions, get useful answers, and act effectively upon what you learn.”
But remember: this can only succeed if you treat Observability as a team sport. That is, everyone in your organization needs to contribute to making your systems observable, coupled with enterprise oversight for tooling decisions, patterns, practices, and so on. You can read more about my take on this topic here.
And now, please enjoy this lovely photo of Barbie hanging out with my husband.
Until next time, peace, love, and code. 🖖💜👩💻
