Published in

Apache Airflow : A Basic Introduction

Fig :- Simplest ETL Pipeline
Fig :- Simple ETL Pipeline Nowadays used in Industry
  1. Error Handling :- Suppose you have ran an ETL task and it fails. Airflow gives us the flexibility to run the task multiple times if it fails to run.
  2. Execution Dependencies :- Suppose You have Ran two Tasks i.e. :- Task A and Task B. Let Task B is Dependent on Task A . So firstly Task A will run then Task B will run.
Fig :- Execution Dependencies
  1. DAG/ Workflow :- Directed Acyclic Graph
Fig :- Directed Acyclic Graph



Data Scientists must think like an artist when finding a solution when creating a piece of code. ⚪️ Artists enjoy working on interesting problems, even if there is no obvious answer ⚪️ 🔵 Follow to join our 28K+ Unique DAILY Readers 🟠

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store