Rethink Metadata … It’s Facets
I’d set context & kicked-off the Rethink Metadata … Blog Series.
Recap — You may be into APIs, Data, ML, Infra, or building tools & frameworks. Regardless, Metadata is intrinsic & plays a vital role in shaping the works.
In this part; let’s explore Enterprise Metadata & It’s Facets.
I’ll be using the words “variant” & “facet” interchangeably throughout this series.
Defining Enterprise Metadata & its Scope
Metadata is often described as “data about data”.
However, I believe Enterprise Metadata goes well beyond data. Information about entities such as Organization, People, Location can also be classified as a type of Enterprise Metadata. Technical details such as information about services, APIs, infrastructure & Operational metrics are also variants of Enterprise Metadata. And details around lifecycles of Data, Tech, Infra, ML are types of Metadata.
To generalize — Enterprise Metadata is intrinsic to a company, and is vital to solving key business problems . Few examples of such business critical problems include — Operational Efficiency, Data Discoverability, Security, Risk management, Compliance, Privacy, ML Ops & Governance. (More about this in next blog).
There are many variants of Metadata. Let move ahead & see …
Facets or Variants of Metadata
An enterprise is a complex super graph of people, technology & process weaved together to solve business problems. Increasingly, companies are putting Data, Technology, AI/ML in the critical path striving to solve the unsolvable. The result is a highly interconnected landscape that is often hard to perceive & leverage in the most effective manner. Over a period of time, industry has developer several highly specialized products & solutions that offer a linear view of individual facets of metadata. But often, these facets are very loosely coupled — limiting the scope of Enterprise Context & its potentials.
Let’s look at a few prominent facets of metadata.
A Deeper Look at Each Facet
Each facet of metadata is very deep in its own nature. Although the ontology is ever evolving, here is an attempt to draw some higher level of hierarchy in each variant of Metadata.
- Data Catalogs & App Catalog are very common in every Data Driven company. They primarily focus on the lifecycle of Data & Apps respectively.
- ML Catalog & Registry are highly relevant in today’s ML driven environments, especially with companies driving business via AI/ML.
- API & Service Catalogs are vital to running the business. These are highly reliable, secure and available systems that cannot afford downtimes.
- Identity & Access is foundational to enabling access to the infrastructure, technologies & data, while keeping them secure.
- Data Classification covers data security & risk management. Data Loss Prevention is a stream that entails scanning & classifying data assets in the company. The resulting metadata is often tied to data catalog.
- Business Metadata covers terminologies, policies, functions & workflows. If you have worked with data governance, you would have come across terms such as “glossary”, “tag”, “stewardship” that fall in this category.
- Org Metadata is part & participle of running a company. Organizations, Locations & People are inherently connected. Workday is classic example.
- Infra Metadata Everything described above runs on some form of infrastructure that is either on the premise, cloud or hybrid. This is the Infra metadata.
Various facets of metadata are deeply interconnected — making a super graph connecting People, Process & Technology. It is just hard to perceive it this way due to the linear views we get today with specialized solutions & products in each field.
Let’s explore the Relevance of Enterprise Metadata in solving key business problems…