Stop the Madness! 1000k data warehouse tables got created out of 6k source tables in 2.5 years Part 2
Solution — 3 Key Principles for Reducing Dimension Tables in your Data Warehouse
If you haven’t read the previous blog, here is a quick link.
Talking to customers with similar symptoms, we realize that this madness of data explosion is a growing but unsolved challenge across industries. We believe that dimensions are the 1st class citizens to be standardized and governed to address the data explosion crisis.
Here are three key principles for managing dimensions summarized from our customers’ real-world practices.
Data Governance: How to Standardize Dimensions
Principle #1 — We strongly encourage the reuse of public dimensions
Public dimension tables possess non-sensitive attributes that are useful and accessible to all business groups. This will help cut off a significant number of duplicated/similar dimension tables.
Principle #2 — We recognize some dimension table variants are valid, but only for special cases. These variants are supposed to be private to their target users.
We allow some teams to use private dimension tables if and only if they have a strong reason for doing so. This has to be carefully reviewed and then to be approved.
Principle #3 — The ultimate goal of standardizing dimensions is to improve CROSS-TEAM COLLABORATION AND ALIGNMENT by speaking “the same language”
A good example of this — Team A would like to compete with Team B on the same KPI. To do so, they have to align the definition of the KPI and make sure the dimension attributes&values are in sync. Otherwise, how would it be possible to benchmark performance in a reasonable level of detail?
What’s next
Topic for the next blog post: Battle-tested strategies for standardising dimensions and metrics.
Leaving feedback:
I’d love to hear back from you on this topic.
RELATED ARTICLE
BI Dashboards are Creating a Technical Debt Black Hole
Stop the Madness! 1000k data warehouse tables got created out of 6k source tables in 2.5 years
Thanks for reading!
Please share, subscribe to my email list, or follow me on Medium for upcoming blogs.
