Really interesting reading. We have struggled with all the described problems. We have been using Airflow for quite some time but it is really a pain to debug and test on your local machine, especially for less technical people. So we ended up to use Airflow only as an orchestrator which do nothing more than starting Docker containers (Snowflake CLI, jupyter, python, etc…) where is all business logic. So the only thing we need to test is that container which is significantly easier but still not perfect. Think Dagster could help us a lot with this, will definitely give it a try!