PinnedSoyel AlaminGroupon Product and EngineeringDr. PySpark: How I Learned to Stop Worrying and Love the Data Pipeline TestingPySpark data-pipeline testing and CICDDec 8, 2020Dec 8, 2020
Soyel AlamTasks Synchronisation in Airflow: Implementing Idempotent DAGs for cyclic dependencyIn an ideal world, An airflow task should represent an atomic transaction. But There could be instances where a task in the current run of…May 12May 12