Observe All The Things
The Importance of Making Data Accessible
What if I told you that there is an entire world out there beyond our tech bubble that could benefit from the power of Observability? Would you believe me?
Consider this definition of Observability, by my awesomely talented friend, Hazel Weakly:
“Observability is the process through which one develops the ability to ask meaningful questions, get useful answers, and act effectively on what you learn.”
I don’t know about you, but that definition sounds like it could definitely apply to stuff outside of tech…don’t you?
Consider the fact that our daily lives are made up of workflows. A workflow is a series of related tasks strung together to accomplish a certain goal. For example:
- Driving your kid(s) to school
- Commuting to and from work
- Getting ready for work in the morning
- Cooking an evening meal
These are examples of “personal”, or “individual” workflows that are unique to you.
There are also “shared” workflows, that affect groups of people. For example:
- Applying for a job
- Onboarding a new employee at a company
- Responding to natural disasters caused by climate change (e.g. forest fires, floods)
- Going to the emergency room (ER)
As an Observability geek, these are the types of workflows that most interest me. Why? Because Observability can help us gain deeper insights into our workflows. By understanding what’s truly happening, we can pinpoint and eliminate bottlenecks, fine-tune processes, and enhance overall efficiency. That is, it enables us to take meaningful action based on what we’ve learned.
Show me one person who hasn’t complained about ER wait times or who hasn’t complained about getting ghosted when applying for a job. Yeah. Exactly. We need this.
So this begs the question…how can we extend Observability beyond our little tech bubble? Luckily for us, we already existing tooling at our disposal:
- OpenTelemetry (OTel) as a standard for generating, collecting, and exporting telemetry data
- Observability backends that help us interpret and the OTel data so that we can make improvements
Okay…but how? Let’s look at an example, shall we?
Going to the emergency room
In Canada, where I live, we are fortunate to have government-provided healthcare; however, we’re certainly not immune to things like long ER wait times. Imagine if we could use Observability to improve ER workflows?
Using OpenTelemetry, we could use traces to capture the end-to-end ER workflow. Spans would represent each step in the workflow:
- intake/registration
- triage
- sitting in the waiting room
- being brought in to an exam area to be seen by a doctor
- waiting in the exam to be seen by a doctor
- speaking to the doctor
- getting tests
- waiting for tests
- speaking to the doctor about test results
- being discharged
We can add attributes to further enrich our spans. For example:
- name of triage nurse
- name of examining physician
- medications prescribed
- tests and/or imaging ordered
We can add logs to capture more detailed information, and correlate them to our spans:
- triage nurse’s notes
- doctor’s exam notes
- discharge notes
- patient feedback
And what about metrics? Span duration can be expressed as a metric. This can help us identify bottlenecks in our workflow. We can also use metrics to provide additional quantitative data about our workflow. For example:
- average time spent in the waiting room by severity of case
- average time spent in the waiting room by time of day, day of the week, time of the year (e.g. holidays vs. non-holidays)
- average nurse and physician shift time
- number of patients per case type (e.g. heart attacks, strokes, car accidents, animal bites)
- number of patients by age group
We can then send the OpenTelemetry data to an Observability backend. This allows us to analyze and interpret the data, and identify areas of improvement, so that decision makers can implement these improvements.
Final Thoughts
As we saw today, we already have standards and tools for gathering telemetry, so why not use them? OpenTelemetry can help us instrument our workflows, and existing Observability backends can help us make sense of the data, allowing us to gain better insights into these workflows, enabling us to improve them.
Before we part ways, I will leave you with one last quote from Hazel Weakly:
“In a world where every team is expected to bring value to the company, in a world where we need to understand systems of ever increasing complexity, why would we POSSIBLY think that we can get away with siloing away our knowledge and understanding our observability systems inside our own companies?”
And now please enjoy a photo of my baby rat, Barbie, taken a couple of days after we got her. She looks very much at home in her little sphere.
Until next time, peace, love, and code. ✌️💜👩💻
Appendix
This blog post is based on a talk that my good friend Marino Wijay and I gave at Observability Day EU 2025. I was supposed to give the talk with my other good friend, Tim Banks, who, unfortunately couldn’t make it to KubeCon. You can watch the recording here:
