Setting up Airflow on Azure & connecting to MS SQL Server
Setting up Airflow on Azure, isn’t quite as easy as on Google Cloud, where Airflow exists as a managed service called “Cloud Composer”.Microsoft has, however, come up with a quick-start template to setup Airflow on Azure, this template sets up both a web-app hosting an Airflow instance and the postgres database backing it up, make it easier to deploy to the cloud.
Unfortunately this web-app is purely based off puckel’s Docker image, without specific configuration applied to it. There is no management of extra dependencies that might be needed, no setup of users and authorizations or to how to deploy dags.
One of the way to fix that is by extending puckels’ image and leverage private container repository. What we will be looking to add in terms of feature:
- Admin login / Access Authorization
- MsSQL Support
- Custom Dag Folders
- Azure Specific interfaces
Use private Azure Container Registry
It is possible to setup a web-app with a container hosted in a private repository, by providing…
