When I joined a project recently which used Airflow, one of the things I found that I was doing was copying the same DAG configuration whenever I needed to create a new DAG.

The following configuration existed in some form in every DAG:

dag = DAG(

We are currently migrating our data warehouse to a new cluster which is GDPR compliant. Historically there hasn’t been a migration strategy in place so we don’t have a source of truth for how our schema’s are structured.

Here is a little query that generates the DDL (Data Definition Language) statements for a whole schema.

Before you run the query create this view.

Hope this helps someone out :)

Alex Handley

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store