Why Cloud Composer Should Be The Only Way You Run AirFlow

After multiple false starts and hours of frustration attempting local and VM installations, I’ve determined the easiest way to run AirFlow is on GCP.

Apache AirFlow is an increasingly in-demand skill for data engineers, but wow it is difficult to install and run, let alone compose and schedule your first direct acyclic graphs (DAGs). In the next few minutes I’ll share why running AirFlow locally is so complex and why Google’s Cloud…

More from Pipeline: A Data Engineering Resource

Offering original and aggregated data engineering content for working and aspiring data professionals. Content posted here generally falls into one of three categories: Technical tutorials, industry news and visualization projects fueled by data engineering.