Airflow: Tips, Tricks, and Pitfalls
Marcin Tustin

Thanks for the great writeup! I’ve just gotten started with Airflow and have a quick question I thought you could help with:

I’m creating dynamic DAGs with the PythonOperator similar to what you have using LogIngesterOperator. Since the task_id is dynamically assigned, which task_id do we pass through to the .set_upstream function to create the actual pipeline? The DAG doesn’t seem to run unless .set_upstream is set. Also, how would you recommend using **kwargs with the PythonOperator when dynamically creating these? Thanks for your time!

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.