What is a Data Angiogram ? How can it help with Data pipeline sanity & Data Quality
I recently had the honour to present at Smartdata 2024 on an idea i came up, related to data quality.
Simple put: How can we test our data pipelines as part of our Continuous integration (CI) ?
What is an Angiogram ?
An angiogram is a medical procedure which one can undergo to find blocks in ones arteries. A block in the artery can lead to a heart attack.
Taking Inspiration from Angiograms, I came up with a Data Angiogram
A data angiogram is a simple technique to assess data pipeline quality and sanity.
I will let the slides take over from here…
A demonstration of the data angiogram for a taxi-ride based data pipeline -mentioned in the slides, is available in the github repo :
Thank you and feel free to reach out to me if you have doubts.
Ps: I am the Chief Inspiration officer at Bytespire.io
happy to help you with your Cost Based Data Engineering needs.
feel to reach out to me at [LinkedIn] or [Twitter] or [X]
Book a free consultation
