Why Building a Self-Service Metrics Platform?
Metrics Store in Action #2–2-minute Tech Tok for 2 years’ Implementation
The previous blog shares how this homegrown self-service metric platform, Pandora, has accelerated time to insight for business users and bent the cost curve on dashboard delivery for IT teams. In this blog, let’s go back to where the story began — what challenges motivated this commercial bank to shift from a traditional BI reporting mindset to a metric standardization mindset in the context of digital transformation.
“Stinky” Fact #1: Metric Consistency
Metric consistency has become one of the most well-known challenges after Airbnb and Uber shared their “recipe” to handle it with the greater data community. There is not much difference from their versions of the story — multiple versions of metrics are defined in different business units for the same or similar business logic. As a result, data users face cognitive overload from having to interpret a multitude of sources of truth, which is obviously not a productive way of working with data in the long run.
Yet, metric discrepancy is not the biggest motivation for a business to kick off the journey toward metric standardization. Any enterprise strategy is 20% about the issue itself and 80% about cost/profit.
“Surprising” Fact #2: Two Hidden Costs of Democratizing Big Data the Old Fashioned Way
In the first blog of this series, we talked about the old-fashioned way to deliver dashboards to business sponsors. In this case, initially, they followed the same path but quickly realized this solution could not scale in their organization with 37384+ staff, 1210+ branches, and petabytes of data.
1. Labor Cost & Person-Months
The cost of hiring enough BI engineers to serve the growing requests from the entire company is way over their budget, assuming 2 person-years ( 1 Data Engineer + 1 Business Analyst ) serves 2 business units.
2. Storage Cost — Data Growth is not 100% Actual Data Growth
Storage cost doesn’t seem much compared to compute cost. But it is a pretty big deal when your company owns petabytes of data, and the growth rate is hundreds of terabytes per day. Putting your feet in their shoes, you’d have to manage the cost. In the old world, business teams used to craft their pipelines and then plug them into the dashboard frontend. Therefore, data is copied and replicated everywhere. This is partially why data is growing faster than your business revenue — data growth is not 100% actual data growth as data copies significantly contribute to the total volume.
Why Building a Self-Service Metric Platform
A Self-Service Metric Platform is created not just to accelerate time to insight and offer a suite of standardized, reusable, and comparable metrics but also to share the data infrastructure and data assets across functions to eliminate unnecessary data copies and engineering efforts.
Quick Links
Enterprise Metric Platform in Action #5
Enterprise Metric Platform in Action #4
Enterprise Metric Platform in Action #3
Enterprise Metric Platform in Action #2
Enterprise Metric Platform in Action #1
Thanks for reading!
If you are interested in this case study, please share, subscribe to my email list, or follow me on Medium for upcoming blogs.
Feel free to share your thoughts in the comments.
