Connect your Data Landscape on Data Vault, Apache NiFi and Agentic AI

Gregor Hohpe sites the following quote for software architecture,

“The set of design decisions about any system that keeps its implementors and maintainers from exercising needless creativity.” — Desmond D’Souza and Alan Cameron Wills

A mark of a successful analytics platform is how easily it is to turn business process (raw) data from disparate data sources into business value. In the previous article in this mini-series we discussed how we can middle out our data strategy on Apache Iceberg and data vault. The fact remains, all data in an organisation is based on three things every enterprise data management (EDM) function manage:

• Its business objects and how to uniquely identify and describe them.
• How those business entities are interacting within business processes, business events and transactions.
• The state of those business entities and interactions and their growth over time.

Software applications record the current state of these things in the technology needed to best serve those applications, we connect to those data sources to retrieve prized critical data elements (CDE) as raw data. Data is raw because it has not yet been curated and classified for those analytical use cases, including AI.