StellarSense
Published in

StellarSense

Rethink Metadata … It’s Facets

Background image by Sander Weeteling on Unsplash

Previously …

I’d set context & kicked-off the Rethink Metadata … Blog Series.

Recap — You may be into APIs, Data, ML, Infra, or building tools & frameworks. Regardless, Metadata is intrinsic & plays a vital role in shaping the works.

In this part; let’s explore Enterprise Metadata & It’s Facets.

I’ll be using the words “variant” & “facet” interchangeably throughout this series.

Defining Enterprise Metadata & its Scope

Metadata is often described as “data about data”.

However, I believe Enterprise Metadata goes well beyond data. Information about entities such as Organization, People, Location can also be classified as a type of Enterprise Metadata. Technical details such as information about services, APIs, infrastructure & Operational metrics are also variants of Enterprise Metadata. And details around lifecycles of Data, Tech, Infra, ML are types of Metadata.

To generalize — Enterprise Metadata is intrinsic to a company, and is vital to solving key business problems . Few examples of such business critical problems include — Operational Efficiency, Data Discoverability, Security, Risk management, Compliance, Privacy, ML Ops & Governance. (More about this in next blog).

There are many variants of Metadata. Let move ahead & see …

Facets or Variants of Metadata

An enterprise is a complex super graph of people, technology & process weaved together to solve business problems. Increasingly, companies are putting Data, Technology, AI/ML in the critical path striving to solve the unsolvable. The result is a highly interconnected landscape that is often hard to perceive & leverage in the most effective manner. Over a period of time, industry has developer several highly specialized products & solutions that offer a linear view of individual facets of metadata. But often, these facets are very loosely coupled — limiting the scope of Enterprise Context & its potentials.

Let’s look at a few prominent facets of metadata.

A Deeper Look at Each Facet

Each facet of metadata is very deep in its own nature. Although the ontology is ever evolving, here is an attempt to draw some higher level of hierarchy in each variant of Metadata.

  • Data Catalogs & App Catalog are very common in every Data Driven company. They primarily focus on the lifecycle of Data & Apps respectively.
  • ML Catalog & Registry are highly relevant in today’s ML driven environments, especially with companies driving business via AI/ML.
  • API & Service Catalogs are vital to running the business. These are highly reliable, secure and available systems that cannot afford downtimes.
  • Identity & Access is foundational to enabling access to the infrastructure, technologies & data, while keeping them secure.
  • Data Classification covers data security & risk management. Data Loss Prevention is a stream that entails scanning & classifying data assets in the company. The resulting metadata is often tied to data catalog.
  • Business Metadata covers terminologies, policies, functions & workflows. If you have worked with data governance, you would have come across terms such as “glossary”, “tag”, “stewardship” that fall in this category.
  • Org Metadata is part & participle of running a company. Organizations, Locations & People are inherently connected. Workday is classic example.
  • Infra Metadata Everything described above runs on some form of infrastructure that is either on the premise, cloud or hybrid. This is the Infra metadata.

Takeaway

Various facets of metadata are deeply interconnected making a super graph connecting People, Process & Technology. It is just hard to perceive it this way due to the linear views we get today with specialized solutions & products in each field.

Up Next…

Let’s explore the Relevance of Enterprise Metadata in solving key business problems…

Background image by Zoltan Tasi on Unsplash

--

--

--

Scaling Enterprise Sense with Deeply Connected Metadata

Recommended from Medium

Getting Started with Beats

A project-driven approach to learning PySpark

Probabilistic Forecasts: Pinball Loss Function

Combinations and Permutations

Dense Mapping Criteria

BigQuery Best Practices

Micro-influencers and NLP

NBA Data: A Live-Example of Data Cleansing

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Deepak Chandramouli

Deepak Chandramouli

https://www.linkedin.com/in/deepakmc/

More from Medium

Rethink Metadata … Gaps & Opportunities

OpenMetadata 0.8.0 Release

A Meta-architecture for Data Mesh

MVDO — Mobile Virtual Data Operator

Left block of two medium boxes bottom one labelled Edge, next one labelled OG Telco, then large box that spans everything labelled CDE — MVDO, under that to the right are three boxes labelled Telco 2, 3 and 4 and below each one of those is a box labelled Edge, above the CDE-MVDO there are five Ecosystem Partners and 16 Customer boxes.