Acing AI
Published in

Acing AI

Data Lineage — An Operational perspective

What can a BI team learn from a data science team?

Photo by Miguel A. Amutio on Unsplash

Data lineage— biggest tool in a business operations toolbox

  • Moving off from an previous application to a newer application: The marketing team decides to move from Marketo to Hubspot. The data perspective Marketo has vs Hubspot is different and this could lead to issues. To solve this problem, it is important to engage with the deployment team for the newer application, prepare a plan and run metrics before and after in different test environments to compare values. Each application has a different view of looking at data points. It is important to meld and have a smooth transition from a metrics perspective before anything process is built on the newer application.
  • Loss of team members: A critical member of the team departs taking a bunch of tribal knowledge about metrics. It is important to have knowledge documentation for different metrics and their definitions stored. Actual data loss could be regenerated by other team members. However, when the metadata context of the data disappears, it becomes a bigger problem. We will discuss the metadata issue solution in our next topic about leverage from data science to business intelligence.
  • Process entropy: A process inevitably transforms data, leading to data becoming muddled. Process design is a critical process. For each process that is designed to fetch values or update values, there should be documentation and communication on the area. Whenever a process changes, a before and after look of the metrics and values should be checked to make sure there are no problems. These processes should be viewed from the purview of a version control so changes could be tied to versions of a process.

What can be leveraged from data science to improve business intelligence?

Importance of a data analytics/BI team

Tools

Recommendations and conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store